-- Logs begin at Wed 2022-07-20 16:48:47 UTC, end at Wed 2022-07-20 18:09:41 UTC. -- Jul 20 17:28:35.650302 dv-cinder-849846-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 17:28:35.657902 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 17:28:35.659895 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 17:28:35 2022] *** Jul 20 17:28:35.659895 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 17:28:35.659895 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 17:28:35.659895 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: nodename: dv-cinder-849846-3-iscsi Jul 20 17:28:35.659895 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: machine: x86_64 Jul 20 17:28:35.659895 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: clock source: unix Jul 20 17:28:35.659895 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: pcre jit disabled Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: detected number of CPU cores: 4 Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: current working directory: / Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: detected binary path: /usr/bin/uwsgi-core Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: your processes number limit is 47680 Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: your memory page size is 4096 bytes Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: detected max file descriptor number: 2048 Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: lock engine: pthread robust mutexes Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: thunder lock: enabled Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 17:28:35.660350 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 17:28:35.705776 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: Python main interpreter initialized at 0x55ed1f7ba9c0 Jul 20 17:28:35.705776 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: python threads support enabled Jul 20 17:28:35.705776 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: your server socket listen backlog is limited to 100 connections Jul 20 17:28:35.705776 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: your mercy for graceful operations on workers is 90 seconds Jul 20 17:28:35.706150 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 17:28:35.706253 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: *** Operational MODE: preforking *** Jul 20 17:28:35.706522 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: *** uWSGI is running in multiple interpreter mode *** Jul 20 17:28:35.706522 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: spawned uWSGI master process (pid: 80570) Jul 20 17:28:35.706719 dv-cinder-849846-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 17:28:35.707819 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: spawned uWSGI worker 1 (pid: 80583, cores: 1) Jul 20 17:28:35.707819 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: spawned uWSGI worker 2 (pid: 80584, cores: 1) Jul 20 17:28:35.707819 dv-cinder-849846-3-iscsi devstack@keystone.service[80570]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 17:28:41.117263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 17:28:41.117263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: from cryptography.utils import int_from_bytes Jul 20 17:28:41.117970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 17:28:41.117970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: from cryptography.utils import int_from_bytes Jul 20 17:28:41.118408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 17:28:41.118408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: from cryptography.utils import int_from_bytes Jul 20 17:28:41.118535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 17:28:41.118535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: from cryptography.utils import int_from_bytes Jul 20 17:28:41.166188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 17:28:41.166374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 17:28:41.166563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] command line args: [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 17:28:41.167230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 17:28:41.167343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 17:28:41.167447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 17:28:41.167533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ================================================================================ {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 17:28:41.167642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] command line args: [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 17:28:41.167898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] admin_token = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.167976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 17:28:41.168074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ================================================================================ {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 17:28:41.168281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] config_dir = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.168365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] admin_token = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.168510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.168597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] config_dir = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.168695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] config_source = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.168772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.168875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] debug = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.168958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] config_source = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.169076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.169386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] debug = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.169386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.169612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.169731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.169832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.169926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] list_limit = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] list_limit = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] log_config_append = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] log_config_append = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.170913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] log_dir = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] log_dir = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] log_file = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] log_file = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.171986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172142 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.172944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.173864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.174024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.174101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.174295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.174384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.174815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.174894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] publish_errors = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] publish_errors = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.175946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] standard_threads = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] standard_threads = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.176967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] use_journal = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] use_journal = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] use_json = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] use_json = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177729 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] use_stderr = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] use_stderr = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] use_syslog = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.177997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] use_syslog = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.178168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.178245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 17:28:41.178430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.178508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.178688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.178765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.178958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.179987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.180068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.180242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.180318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.180521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.180602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.180789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.180873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.181894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.182061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.182138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.182326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.182435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.183133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.183220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.183396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.183474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.183659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.183737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.183921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.184990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.185954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.186969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.187993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.188178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.188254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.188482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.188554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.188749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.188817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.188983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.189928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.190121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.190256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.190367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.190525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.190602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.190816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.190905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.191979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.192946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.193888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.194973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.195927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.196882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.197891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.198853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.199951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.200980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.201894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.202965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.203993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] auth.external = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.204977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] auth.password = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] auth.token = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] auth.external = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.205900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] auth.password = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] auth.token = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.206998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] credential.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.207951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] credential.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.208919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.209961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.210961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.211915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.212900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.213860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214010 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.214969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.215968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216047 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.216950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.217940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.218986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.219918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.220875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.221973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.222978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.223902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.224863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225019 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.225954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.226969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.227906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.228856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.229002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.229080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.229736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.229829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.229976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.230986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.231927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.232903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.233981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.234988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.235942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.236893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] role.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.237992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] role.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.238934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] role.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] role.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.239964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240623 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.240954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.241964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.242972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.243944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.244956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.245629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.245710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.245863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.245943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.246872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.247993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.248948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249722 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.249963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250126 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.250897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.connection = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.251861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.connection = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.252984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.253925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.254922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.255078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.255157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.255308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.255388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.255478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80583) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 17:28:41.255557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.255770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.255999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.256239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.256493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 17:28:41.256666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80584) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 17:28:41.286078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.286293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.286387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.286526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.286615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.286748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.295360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.295680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.323795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.324026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.324113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.324316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.465193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.465449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.513968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.514199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.514719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.514955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.577818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.578998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.579141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.579360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.603604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.603835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.603954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.604086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.604229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.604387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.604529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.604657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.604779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.604918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.605156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.605275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.605275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.615838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.616082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.628925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.629154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.629341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.629426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.629578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.629661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.629791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.629873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.630005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.630097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.630218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80583) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.630301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.630430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.630647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80584) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 17:28:41.647133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.647345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 17:28:41.647461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.647679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 17:28:41.647788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.647867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 17:28:41.648008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.648127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.648210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 17:28:41.648388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.648490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.648674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.648768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.648950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.649040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.649234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.660710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.660987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.661071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.661254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.661336 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.661534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.661616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.661868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.669690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.670373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.672938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.673221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.673501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.673652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.673897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.673978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.674180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.674269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.674439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.674572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.674706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.674857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.674979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.675119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.675248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.675389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.675517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.675662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.675782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.675934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.676205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.676496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.702434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.703648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.704025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.704303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.705277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.705559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.709529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.709807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.710084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.710787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.711066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.711347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.717878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.718111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.719188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.719472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.722679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.722963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.723239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.723520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.723790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.724058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.724147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.724324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.724405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.724614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.724694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.724892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.725167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.725440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.725705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.725970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.743131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.743408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.745528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.745805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.748810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.750347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.752410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.752727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.753998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.754294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.758017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.759302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.762175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.763483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.766227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.766520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.766808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.767093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.767277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.767383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.767574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.767656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.767866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.768164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.768462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.768744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.786913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.787273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.787539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.787808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.787894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.788252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.788556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.788832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.795690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.796802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.797402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.797672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.798496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.798769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.800720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.800999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.801278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.801560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.801838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.801919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.802174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.802456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.802737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.803016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.815601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.815877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.816154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.816462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.816742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.816873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.817031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.817152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.817306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.817434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.817715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.817996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.818293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.818570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.836947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.838316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.840208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.841637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.843504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.844940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.845131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.846545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.846750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.847026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.847306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.848178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.848481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.848770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.854803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.856210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.856569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.857844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.859791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.860080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.860364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.860672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.861067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.861356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.861641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.861917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.870352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.870661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.870941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.871646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.871934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.872214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.878741 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80583) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.879016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.879290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.879561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.879827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.880055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80584) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 17:28:41.880137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.880334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.880424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.880637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.880717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.880913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.880991 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.881179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.881269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.881450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.881532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.881708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.881788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.881967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.882048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.882227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80583) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.882306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.882495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.882770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.883030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.883303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.883574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80584) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 17:28:41.921100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ed1f7ba9c0 pid: 80583 (default app) Jul 20 17:28:41.922421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ed1f7ba9c0 pid: 80584 (default app) Jul 20 17:28:41.924371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad880f7-157c-4d45-bd6b-18dabb8a3646 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:41.924690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad880f7-157c-4d45-bd6b-18dabb8a3646 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:41.924969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad880f7-157c-4d45-bd6b-18dabb8a3646 None None] PATH_INFO: `/v3` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:41.929192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed Jul 20 17:28:41 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:28:50.576034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7216-7bb3-4eb6-9964-1a9e929a1dc9 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:50.576610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7216-7bb3-4eb6-9964-1a9e929a1dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:50.576610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e7216-7bb3-4eb6-9964-1a9e929a1dc9 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:50.580916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:28:50.586111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:50.586534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:50.586666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:50.655240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: INFO dbcounter [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] Registered counter for database keystone Jul 20 17:28:50.676796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG oslo_db.sqlalchemy.engines [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80583) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 17:28:50.683028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writer thread running {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 17:28:50.859736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG passlib.handlers.bcrypt [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80583) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 17:28:50.860066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG passlib.handlers.bcrypt [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80583) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 17:28:50.914280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:28:50.915888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:28:51.036530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:28:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 20 17:28:51.223073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e7216-7bb3-4eb6-9964-1a9e929a1dc9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:51.293263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: INFO dbcounter [None req-7d1e7216-7bb3-4eb6-9964-1a9e929a1dc9 None None] Registered counter for database keystone Jul 20 17:28:51.312161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG oslo_db.sqlalchemy.engines [None req-7d1e7216-7bb3-4eb6-9964-1a9e929a1dc9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80584) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 17:28:51.318273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writer thread running {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 17:28:51.436647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e7216-7bb3-4eb6-9964-1a9e929a1dc9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:51.440924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ea19b-a494-46d3-887e-0194c36bb933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:51.441588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ea19b-a494-46d3-887e-0194c36bb933 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:51.441872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ea19b-a494-46d3-887e-0194c36bb933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:51.442147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ea19b-a494-46d3-887e-0194c36bb933 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:51.450367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ea19b-a494-46d3-887e-0194c36bb933 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:51.502139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING keystone.common.rbac_enforcer.enforcer [None req-158ea19b-a494-46d3-887e-0194c36bb933 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 17:28:51.502421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ea19b-a494-46d3-887e-0194c36bb933 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:51.511913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Jul 20 17:28:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 17:28:51.518002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:51.533456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f3797-ea28-4673-bbb2-83ec6a2eee45 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:51.537948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afe9e3-30d4-4bb9-92f6-9d60c95430c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:51.548238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afe9e3-30d4-4bb9-92f6-9d60c95430c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:51.548238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afe9e3-30d4-4bb9-92f6-9d60c95430c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:51.548238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afe9e3-30d4-4bb9-92f6-9d60c95430c1 admin admin] PATH_INFO: `/v3/projects/3ddf29534d9849ed9221c0b30b65b542` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:51.548238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afe9e3-30d4-4bb9-92f6-9d60c95430c1 admin admin] RBAC: Authorizing `identity:get_project(project_id=3ddf29534d9849ed9221c0b30b65b542)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:51.595980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING keystone.common.rbac_enforcer.enforcer [None req-37afe9e3-30d4-4bb9-92f6-9d60c95430c1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 17:28:51.596538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afe9e3-30d4-4bb9-92f6-9d60c95430c1 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:51.600931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Jul 20 17:28:51 2022] GET /identity/v3/projects/3ddf29534d9849ed9221c0b30b65b542 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:28:53.606711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd68bf-fac4-418e-b1cd-e35bafb05b03 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:53.607219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd68bf-fac4-418e-b1cd-e35bafb05b03 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:53.607324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd68bf-fac4-418e-b1cd-e35bafb05b03 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:53.611376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:28:53.616700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c892-78fe-4abd-81d6-bddb55bf9b3b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:53.617072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c892-78fe-4abd-81d6-bddb55bf9b3b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:53.617281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c892-78fe-4abd-81d6-bddb55bf9b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:53.636783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-9f33c892-78fe-4abd-81d6-bddb55bf9b3b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:28:53.638318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9f33c892-78fe-4abd-81d6-bddb55bf9b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:28:53.646259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:28:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 17:28:53.668422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd68bf-fac4-418e-b1cd-e35bafb05b03 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:53.680398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd68bf-fac4-418e-b1cd-e35bafb05b03 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:53.685152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67853a-4c2c-449a-8264-6ea1a9004799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:53.685789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67853a-4c2c-449a-8264-6ea1a9004799 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:53.686086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67853a-4c2c-449a-8264-6ea1a9004799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:53.686368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67853a-4c2c-449a-8264-6ea1a9004799 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:53.694069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67853a-4c2c-449a-8264-6ea1a9004799 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:53.695858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67853a-4c2c-449a-8264-6ea1a9004799 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:53.705257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Jul 20 17:28:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:28:53.710543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33c892-78fe-4abd-81d6-bddb55bf9b3b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:53.720154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33c892-78fe-4abd-81d6-bddb55bf9b3b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:53.724562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58790c-23fc-4284-b0eb-17019609600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:53.725419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58790c-23fc-4284-b0eb-17019609600b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:53.725710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58790c-23fc-4284-b0eb-17019609600b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:53.725990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58790c-23fc-4284-b0eb-17019609600b admin admin] PATH_INFO: `/v3/users/85af2924b4aa4547ad22ecda72bbb229` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:53.727401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58790c-23fc-4284-b0eb-17019609600b admin admin] RBAC: Authorizing `identity:get_user(user_id=85af2924b4aa4547ad22ecda72bbb229)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:53.729877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58790c-23fc-4284-b0eb-17019609600b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:53.731453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed Jul 20 17:28:53 2022] GET /identity/v3/users/85af2924b4aa4547ad22ecda72bbb229 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:28:55.803833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d036ca-8fc4-44f7-8e8c-5fa47e64a5d0 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:55.804369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d036ca-8fc4-44f7-8e8c-5fa47e64a5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:55.804369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d036ca-8fc4-44f7-8e8c-5fa47e64a5d0 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:55.805767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:28:55.811471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4e8c0-8eb3-48e5-8958-0274f97589c7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:55.811864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4e8c0-8eb3-48e5-8958-0274f97589c7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:55.812035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4e8c0-8eb3-48e5-8958-0274f97589c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:55.828927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-2bb4e8c0-8eb3-48e5-8958-0274f97589c7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:28:55.830096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2bb4e8c0-8eb3-48e5-8958-0274f97589c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:28:55.850832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353edf0-0841-4915-8df2-ade570b3d702 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:55.851119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353edf0-0841-4915-8df2-ade570b3d702 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:55.851401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353edf0-0841-4915-8df2-ade570b3d702 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:55.852894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:28:55.858008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:55.858297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:55.858576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:55.897946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG passlib.handlers.bcrypt [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80584) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 17:28:55.898356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG passlib.handlers.bcrypt [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80584) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 17:28:55.910052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:28:55.933261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4e8c0-8eb3-48e5-8958-0274f97589c7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:55.945150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:28:55.946770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:28:55.956659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:56.036200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4e8c0-8eb3-48e5-8958-0274f97589c7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.044192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:28:56.066544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:56.069156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.069961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.070247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.070555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.087257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:28:56.098815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:28:56.099875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:28:56.100871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:28:56.106232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:56.179609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34206537-f0fa-463e-8c6f-3580b2f4ee5c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.182473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:56.189815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:56.207406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb97bd1-9ab1-41b3-819a-63800c90c8cf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.216820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.217705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.217998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.218271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.241506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.242173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.242455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.242731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.243483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:28:56.243791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:56.245319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:56.256768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:28:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:28:56.262767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:56.286591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:56.288285 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:56.359877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6692a7-a2ff-44dc-9d67-13bcb283a9be None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.384181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:28:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 17:28:56.390565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:56.393200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:56.393312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.394068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.394359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.394636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] PATH_INFO: `/v3/users/85af2924b4aa4547ad22ecda72bbb229` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.395864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] RBAC: Authorizing `identity:get_user(user_id=85af2924b4aa4547ad22ecda72bbb229)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:56.397370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:56.399004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:28:56 2022] GET /identity/v3/users/85af2924b4aa4547ad22ecda72bbb229 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:28:56.404379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:56.419212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:56.486523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19169831-2eca-4660-a178-2ab60cf49f8a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.502692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd10c3-a1c8-48fe-8bf0-612d1d6588dc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.520319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.521143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.521436 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.521715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.535686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.536505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.536791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.537065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.538233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:56.539675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:56.541840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:28:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:28:56.542217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:56.543883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:56.547555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:28:56.564174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:28:56.565214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:28:56.566288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:28:56.566288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:28:56.566288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:28:56.566288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:28:56.566288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:28:56.570932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:56.626354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:56.628526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:56.646406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eed5e-67c6-44b4-90ef-ded8db50234f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.669740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaf90-850c-4162-bb49-f88c091cfc72 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:56.680065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bef207-f279-4e13-8d38-74d3e02e6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.680771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bef207-f279-4e13-8d38-74d3e02e6b8d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.681306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bef207-f279-4e13-8d38-74d3e02e6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.681769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bef207-f279-4e13-8d38-74d3e02e6b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.683528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bef207-f279-4e13-8d38-74d3e02e6b8d None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.domain.id=default, user.id=85af2924b4aa4547ad22ecda72bbb229) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:28:56.683851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bef207-f279-4e13-8d38-74d3e02e6b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:56.685334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bef207-f279-4e13-8d38-74d3e02e6b8d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:56.691929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Jul 20 17:28:56 2022] GET /identity/v3/role_assignments?user.id=85af2924b4aa4547ad22ecda72bbb229&scope.domain.id=default&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:28:56.703967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c15fb6-9420-4206-8a26-232c6ad6bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:56.704696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c15fb6-9420-4206-8a26-232c6ad6bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:56.704983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c15fb6-9420-4206-8a26-232c6ad6bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:56.705259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c15fb6-9420-4206-8a26-232c6ad6bdc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:56.706015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c15fb6-9420-4206-8a26-232c6ad6bdc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:28:56.706325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c15fb6-9420-4206-8a26-232c6ad6bdc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:56.707774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c15fb6-9420-4206-8a26-232c6ad6bdc1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:56.727723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:28:59.162370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f6b14-1979-45e9-96e3-7e0dfd780c5c None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.162862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f6b14-1979-45e9-96e3-7e0dfd780c5c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.162958 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f6b14-1979-45e9-96e3-7e0dfd780c5c None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.164315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:28:59.170205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2407e-719b-46ad-83ec-abb4372c919f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.170626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2407e-719b-46ad-83ec-abb4372c919f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.170797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2407e-719b-46ad-83ec-abb4372c919f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.194736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c2a2407e-719b-46ad-83ec-abb4372c919f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:28:59.196180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c2a2407e-719b-46ad-83ec-abb4372c919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:28:59.248656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a525c-0ce3-4ce8-ac57-7ffe6cb34a00 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.249193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a525c-0ce3-4ce8-ac57-7ffe6cb34a00 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.249701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a525c-0ce3-4ce8-ac57-7ffe6cb34a00 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.251643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 13/23] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:28:59.257036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb298-d220-46bf-b06b-d9587baa093a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.258074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb298-d220-46bf-b06b-d9587baa093a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.258566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb298-d220-46bf-b06b-d9587baa093a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.280360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 11/24] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:28:59.289457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b52cb298-d220-46bf-b06b-d9587baa093a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:28:59.290911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b52cb298-d220-46bf-b06b-d9587baa093a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:28:59.322020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2407e-719b-46ad-83ec-abb4372c919f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:59.410417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 14/25] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:28:59.432796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:59.448733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb298-d220-46bf-b06b-d9587baa093a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:59.560575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2407e-719b-46ad-83ec-abb4372c919f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:59.627431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed84e7-ec73-446a-9c6d-a291fd736f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:59.628213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:59.628872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed84e7-ec73-446a-9c6d-a291fd736f0c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.629352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed84e7-ec73-446a-9c6d-a291fd736f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.629819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed84e7-ec73-446a-9c6d-a291fd736f0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.654622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed84e7-ec73-446a-9c6d-a291fd736f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-33ed84e7-ec73-446a-9c6d-a291fd736f0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:28:59.664286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:28:59.665152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:28:59.665923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:28:59.675848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb298-d220-46bf-b06b-d9587baa093a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:59.685228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 12/26] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:28:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:28:59.686855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da5d38-c43e-41ce-b36d-25caf3e2ec56 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.687706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da5d38-c43e-41ce-b36d-25caf3e2ec56 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.688236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da5d38-c43e-41ce-b36d-25caf3e2ec56 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.690038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 13/27] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:28:59.694682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da5d38-c43e-41ce-b36d-25caf3e2ec56 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:59.738415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c02f-f83d-42b6-8ce3-5c63497aa5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:59.739573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c02f-f83d-42b6-8ce3-5c63497aa5de None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.740051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c02f-f83d-42b6-8ce3-5c63497aa5de None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.740547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c02f-f83d-42b6-8ce3-5c63497aa5de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.757762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6c02f-f83d-42b6-8ce3-5c63497aa5de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:59.759598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6c02f-f83d-42b6-8ce3-5c63497aa5de None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-3fc6c02f-f83d-42b6-8ce3-5c63497aa5de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:28:59.783923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:28:59.784809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:28:59.785633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:28:59.785633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:28:59.785633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:28:59.785633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:28:59.787550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 15/28] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:28:59.789117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0842a-920b-4918-a1b0-2ca98b821824 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.789600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0842a-920b-4918-a1b0-2ca98b821824 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.790078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0842a-920b-4918-a1b0-2ca98b821824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.819428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-67c0842a-920b-4918-a1b0-2ca98b821824 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:28:59.821012 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-67c0842a-920b-4918-a1b0-2ca98b821824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:28:59.825243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da5d38-c43e-41ce-b36d-25caf3e2ec56 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:28:59.837012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:28:59.876609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:28:59.877693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.877982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.878265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.879041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:28:59.879375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:28:59.881621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:28:59.898868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 14/29] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:28:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:28:59.902134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:28:59.919083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 16/30] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:28:59.920790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43abd0-3934-4428-b0ac-a965c766ca21 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:28:59.921289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43abd0-3934-4428-b0ac-a965c766ca21 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:28:59.921767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43abd0-3934-4428-b0ac-a965c766ca21 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:28:59.923384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 17/31] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:28:59.924306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43abd0-3934-4428-b0ac-a965c766ca21 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:00.020912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:00.045761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61b139-eff5-42e6-b1c4-a675090b9cd2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:00.081344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43abd0-3934-4428-b0ac-a965c766ca21 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:00.082225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993bc9-90e0-427b-acf9-d9018a408b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:00.083192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993bc9-90e0-427b-acf9-d9018a408b55 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.083679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993bc9-90e0-427b-acf9-d9018a408b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.084160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993bc9-90e0-427b-acf9-d9018a408b55 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.085130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993bc9-90e0-427b-acf9-d9018a408b55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:00.085674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993bc9-90e0-427b-acf9-d9018a408b55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:00.087288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993bc9-90e0-427b-acf9-d9018a408b55 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:00.106487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 15/32] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:00.108719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670256-472c-4e37-b891-346c98db7b29 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.109223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670256-472c-4e37-b891-346c98db7b29 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.109725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670256-472c-4e37-b891-346c98db7b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.131324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:00.132205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.132526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.132806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] PATH_INFO: `/v3/users/85af2924b4aa4547ad22ecda72bbb229` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.133680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-47670256-472c-4e37-b891-346c98db7b29 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:00.134009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] RBAC: Authorizing `identity:get_user(user_id=85af2924b4aa4547ad22ecda72bbb229)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:00.136206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:00.137321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 18/33] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:28:59 2022] GET /identity/v3/users/85af2924b4aa4547ad22ecda72bbb229 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:00.137894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:00.138276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-47670256-472c-4e37-b891-346c98db7b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:00.170910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:00.225156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 16/34] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:00.226018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47670256-472c-4e37-b891-346c98db7b29 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:00.263030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c26f62-f1d9-4846-b7ed-9a0714ea75d9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:00.287563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:00.298742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:00.299355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.302156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.302156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.302156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:00.302156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:00.339783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47670256-472c-4e37-b891-346c98db7b29 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:00.373374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e73fb-44d6-402f-afb1-f4a3da280b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:00.374042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e73fb-44d6-402f-afb1-f4a3da280b49 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.374338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e73fb-44d6-402f-afb1-f4a3da280b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.374618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e73fb-44d6-402f-afb1-f4a3da280b49 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.375570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e73fb-44d6-402f-afb1-f4a3da280b49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:00.377080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e73fb-44d6-402f-afb1-f4a3da280b49 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:00.511378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 19/35] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jul 20 17:29:00 2022] POST /identity/v3/roles => generated 219 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:00.512296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:00.600702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 17/36] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:29:00 2022] POST /identity/v3/projects => generated 301 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:00.602624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f21ca2-e137-403a-92cf-00f5be4d95ff None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.603065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f21ca2-e137-403a-92cf-00f5be4d95ff None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.603489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f21ca2-e137-403a-92cf-00f5be4d95ff None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.605684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 18/37] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:00.606739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f21ca2-e137-403a-92cf-00f5be4d95ff None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:00.624478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:00.701352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231696b-2629-4e0e-9424-cef7217a2fcc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:00.745802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046fdf3-4e8e-43d2-a469-17b7694b291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:00.746285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046fdf3-4e8e-43d2-a469-17b7694b291a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.746581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046fdf3-4e8e-43d2-a469-17b7694b291a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.746880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046fdf3-4e8e-43d2-a469-17b7694b291a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.749051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046fdf3-4e8e-43d2-a469-17b7694b291a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:00.751333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046fdf3-4e8e-43d2-a469-17b7694b291a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:00.753642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 20/38] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:00.755197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad3e14-c3ac-40ab-909c-c3c0f3daecca None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.755501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad3e14-c3ac-40ab-909c-c3c0f3daecca None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.755788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad3e14-c3ac-40ab-909c-c3c0f3daecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.774925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f21ca2-e137-403a-92cf-00f5be4d95ff None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:00.779659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e3ad3e14-c3ac-40ab-909c-c3c0f3daecca None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:00.782987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e3ad3e14-c3ac-40ab-909c-c3c0f3daecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:00.829605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10e660-db4b-4cda-b7a8-81cb2c22a03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:00.830324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e660-db4b-4cda-b7a8-81cb2c22a03a None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.830615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e660-db4b-4cda-b7a8-81cb2c22a03a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.830897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10e660-db4b-4cda-b7a8-81cb2c22a03a None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.831666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10e660-db4b-4cda-b7a8-81cb2c22a03a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:00.833904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10e660-db4b-4cda-b7a8-81cb2c22a03a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:00.849579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:00.870087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 21/39] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:00.871212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad3e14-c3ac-40ab-909c-c3c0f3daecca None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:00.911121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 19/40] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jul 20 17:29:00 2022] POST /identity/v3/roles => generated 225 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:00.912458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20fbbe-4bb7-420e-9053-471df2a93dbc None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:00.912836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20fbbe-4bb7-420e-9053-471df2a93dbc None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:00.913521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20fbbe-4bb7-420e-9053-471df2a93dbc None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:00.915109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 20/41] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:00.915888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20fbbe-4bb7-420e-9053-471df2a93dbc None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:00.998325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad3e14-c3ac-40ab-909c-c3c0f3daecca None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:00.998867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:01.035025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20fbbe-4bb7-420e-9053-471df2a93dbc None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:01.037238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7259442-a36a-455b-bab7-1742c6bafa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:01.038136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259442-a36a-455b-bab7-1742c6bafa7b None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:01.038709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259442-a36a-455b-bab7-1742c6bafa7b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:01.038709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259442-a36a-455b-bab7-1742c6bafa7b None admin] PATH_INFO: `/v3/domains/default/users/85af2924b4aa4547ad22ecda72bbb229/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:01.040842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7259442-a36a-455b-bab7-1742c6bafa7b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=85af2924b4aa4547ad22ecda72bbb229, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:01.042963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7259442-a36a-455b-bab7-1742c6bafa7b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:01.072558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:01.073229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:01.073516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:01.073802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:01.074587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:01.075990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:01.149731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 22/42] 10.222.0.42 () {58 vars in 1361 bytes} [Wed Jul 20 17:29:00 2022] PUT /identity/v3/domains/default/users/85af2924b4aa4547ad22ecda72bbb229/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:01.151523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a591a8a-0ad7-45a9-9099-83fb6fda09e0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:01.151801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a591a8a-0ad7-45a9-9099-83fb6fda09e0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:01.152089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a591a8a-0ad7-45a9-9099-83fb6fda09e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:01.180244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 21/43] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jul 20 17:29:00 2022] POST /identity/v3/roles => generated 223 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:01.281156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0a591a8a-0ad7-45a9-9099-83fb6fda09e0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:01.282698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0a591a8a-0ad7-45a9-9099-83fb6fda09e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:01.327296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:01.360001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 23/44] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:01.383207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:01.485968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490b0c1-c369-439b-a048-38c30b167bbc None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:01.513939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:01.519770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7c93c-ffeb-4a0e-b50e-c54fc50b94eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:01.520603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7c93c-ffeb-4a0e-b50e-c54fc50b94eb None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:01.520891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7c93c-ffeb-4a0e-b50e-c54fc50b94eb None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:01.521170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7c93c-ffeb-4a0e-b50e-c54fc50b94eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:01.522458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7c93c-ffeb-4a0e-b50e-c54fc50b94eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:01.523927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7c93c-ffeb-4a0e-b50e-c54fc50b94eb None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:01.525777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 22/45] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:01.531124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a591a8a-0ad7-45a9-9099-83fb6fda09e0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:01.628104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a591a8a-0ad7-45a9-9099-83fb6fda09e0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:01.661782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8abb2-d82a-4367-b2d6-b04eb39052c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:01.662443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8abb2-d82a-4367-b2d6-b04eb39052c4 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:01.662727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8abb2-d82a-4367-b2d6-b04eb39052c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:01.663007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8abb2-d82a-4367-b2d6-b04eb39052c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:01.663727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8abb2-d82a-4367-b2d6-b04eb39052c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:01.665223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c8abb2-d82a-4367-b2d6-b04eb39052c4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:01.750341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:29:01 2022] POST /identity/v3/projects => generated 312 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:03.211901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce19f5c-e730-467c-abfe-ec335e8ef754 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.212396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce19f5c-e730-467c-abfe-ec335e8ef754 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.212523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce19f5c-e730-467c-abfe-ec335e8ef754 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.214444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:03.219800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc15e7-7a38-4e05-a1e8-5a0bb8f39b6c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.220218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc15e7-7a38-4e05-a1e8-5a0bb8f39b6c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.220375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc15e7-7a38-4e05-a1e8-5a0bb8f39b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.236903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5ccc15e7-7a38-4e05-a1e8-5a0bb8f39b6c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:03.238094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5ccc15e7-7a38-4e05-a1e8-5a0bb8f39b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:03.254843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:03.303530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:03.327387 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce19f5c-e730-467c-abfe-ec335e8ef754 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:03.427112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce19f5c-e730-467c-abfe-ec335e8ef754 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:03.460273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:03.461095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.461388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.461672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.471961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:03.479655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:03.480675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:03.481478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:03.486670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc15e7-7a38-4e05-a1e8-5a0bb8f39b6c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:03.583779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc15e7-7a38-4e05-a1e8-5a0bb8f39b6c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:03.616400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266f3f2-a276-4487-8b9b-d59d8cdda17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:03.617095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f3f2-a276-4487-8b9b-d59d8cdda17a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.617387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f3f2-a276-4487-8b9b-d59d8cdda17a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.617730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f3f2-a276-4487-8b9b-d59d8cdda17a None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.618496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266f3f2-a276-4487-8b9b-d59d8cdda17a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:03.618811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266f3f2-a276-4487-8b9b-d59d8cdda17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:03.620837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266f3f2-a276-4487-8b9b-d59d8cdda17a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:03.629495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 26/50] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:03.635625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:03.693723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:03.734984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221b66f-437d-46fd-8fe8-378685da2b7f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:03.768077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:03.768976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.769266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.769546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] PATH_INFO: `/v3/users/85af2924b4aa4547ad22ecda72bbb229` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.770968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=85af2924b4aa4547ad22ecda72bbb229)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:03.772439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:03.776486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 25/51] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:03 2022] GET /identity/v3/users/85af2924b4aa4547ad22ecda72bbb229 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:03.776632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6674b4-9067-49aa-8511-94f75532ca7a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.776632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6674b4-9067-49aa-8511-94f75532ca7a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.776632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6674b4-9067-49aa-8511-94f75532ca7a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.777391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 27/52] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:03.779941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:03.782585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522ac4-fd9d-4ec3-b8ac-cbb13c3526cc None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.782882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522ac4-fd9d-4ec3-b8ac-cbb13c3526cc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.783173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522ac4-fd9d-4ec3-b8ac-cbb13c3526cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.799356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b2522ac4-fd9d-4ec3-b8ac-cbb13c3526cc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:03.800600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b2522ac4-fd9d-4ec3-b8ac-cbb13c3526cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:03.811036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:03.865895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 28/53] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:03.877220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e569e-93f8-448a-bccd-92c7cb183fb4 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:03.889481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2522ac4-fd9d-4ec3-b8ac-cbb13c3526cc None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:03.901951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:03.910433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:03.911286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:03.911650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:03.911878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:03.913097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:03.914593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:03.917710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 26/54] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:03.922423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:03.990852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2522ac4-fd9d-4ec3-b8ac-cbb13c3526cc None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:04.027161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:04.027986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:04.028276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:04.028636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:04.029859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:04.031347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:04.033317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 29/55] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:04.038928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:04.054244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:04.133574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbcd86-5d29-4c14-b2ed-960b90b6a5db None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:04.142610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943743a-6bcc-41f1-8d0d-9a5adb4af5d6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:04.177399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1eef8-6d68-4357-9337-6d65b04ce908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:04.178050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1eef8-6d68-4357-9337-6d65b04ce908 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:04.178342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1eef8-6d68-4357-9337-6d65b04ce908 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:04.178625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1eef8-6d68-4357-9337-6d65b04ce908 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:04.179201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977b64-c3e1-4e7f-91cb-447e7a285b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:04.179354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1eef8-6d68-4357-9337-6d65b04ce908 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:04.179859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977b64-c3e1-4e7f-91cb-447e7a285b9b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:04.180144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977b64-c3e1-4e7f-91cb-447e7a285b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:04.180421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977b64-c3e1-4e7f-91cb-447e7a285b9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:04.180820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1eef8-6d68-4357-9337-6d65b04ce908 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:04.181289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977b64-c3e1-4e7f-91cb-447e7a285b9b None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.domain.id=default, user.id=85af2924b4aa4547ad22ecda72bbb229) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:04.182213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977b64-c3e1-4e7f-91cb-447e7a285b9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:04.183775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977b64-c3e1-4e7f-91cb-447e7a285b9b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:04.203015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 27/56] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Jul 20 17:29:03 2022] GET /identity/v3/role_assignments?user.id=85af2924b4aa4547ad22ecda72bbb229&scope.domain.id=default&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 530 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:04.299101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Jul 20 17:29:04 2022] POST /identity/v3/projects => generated 298 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:29:06.336003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda10b5-e9e6-4fb3-9e0f-7fcd520e2182 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:06.336534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda10b5-e9e6-4fb3-9e0f-7fcd520e2182 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:06.336619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda10b5-e9e6-4fb3-9e0f-7fcd520e2182 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:06.338491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:06.343769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11aae2c-460e-406e-8f7c-19d4bd872334 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:06.344157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11aae2c-460e-406e-8f7c-19d4bd872334 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:06.344340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11aae2c-460e-406e-8f7c-19d4bd872334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:06.361237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d11aae2c-460e-406e-8f7c-19d4bd872334 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:06.362385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d11aae2c-460e-406e-8f7c-19d4bd872334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:06.419664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:06.426918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:06.450462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda10b5-e9e6-4fb3-9e0f-7fcd520e2182 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:06.509119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:06.550445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda10b5-e9e6-4fb3-9e0f-7fcd520e2182 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:06.582496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bf28e-28e1-4f1b-a74a-bfa59d2c092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:06.583278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf28e-28e1-4f1b-a74a-bfa59d2c092a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:06.583568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf28e-28e1-4f1b-a74a-bfa59d2c092a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:06.583846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf28e-28e1-4f1b-a74a-bfa59d2c092a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:06.585040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bf28e-28e1-4f1b-a74a-bfa59d2c092a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:06.586487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bf28e-28e1-4f1b-a74a-bfa59d2c092a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:06.588683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:06.594060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11aae2c-460e-406e-8f7c-19d4bd872334 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:06.687493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11aae2c-460e-406e-8f7c-19d4bd872334 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:06.720135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a78e2c-0d81-4677-b8c4-2608945c16ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:06.720897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a78e2c-0d81-4677-b8c4-2608945c16ab None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:06.721179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a78e2c-0d81-4677-b8c4-2608945c16ab None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:06.721454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a78e2c-0d81-4677-b8c4-2608945c16ab None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:06.722172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a78e2c-0d81-4677-b8c4-2608945c16ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:06.723598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a78e2c-0d81-4677-b8c4-2608945c16ab None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:06.828861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:29:06 2022] POST /identity/v3/users => generated 275 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:09.970230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a34b-ca9b-45a7-b42a-f46926b8d397 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:09.970735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a34b-ca9b-45a7-b42a-f46926b8d397 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:09.970807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a34b-ca9b-45a7-b42a-f46926b8d397 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:09.972345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:09.977622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25991eba-19ab-4dd6-907f-aa50af3727e3 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:09.978005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25991eba-19ab-4dd6-907f-aa50af3727e3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:09.978182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25991eba-19ab-4dd6-907f-aa50af3727e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:09.994203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f27a1-72cf-4a69-9f51-3ca67170fa3b None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:09.994846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f27a1-72cf-4a69-9f51-3ca67170fa3b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:09.995356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f27a1-72cf-4a69-9f51-3ca67170fa3b None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:09.996980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 31/63] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:10.001435 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-25991eba-19ab-4dd6-907f-aa50af3727e3 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:10.002190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c024a5f-4ec7-4317-82f0-a24d1cf9802b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.002486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c024a5f-4ec7-4317-82f0-a24d1cf9802b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.002772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c024a5f-4ec7-4317-82f0-a24d1cf9802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.003521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-25991eba-19ab-4dd6-907f-aa50af3727e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:10.025510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-2c024a5f-4ec7-4317-82f0-a24d1cf9802b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:10.027388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2c024a5f-4ec7-4317-82f0-a24d1cf9802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:10.077092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:10.105834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 32/64] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:10.126354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 33/65] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:10.151108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c024a5f-4ec7-4317-82f0-a24d1cf9802b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.167024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25991eba-19ab-4dd6-907f-aa50af3727e3 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.232899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:10.329508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c024a5f-4ec7-4317-82f0-a24d1cf9802b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:10.332938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25991eba-19ab-4dd6-907f-aa50af3727e3 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:10.382503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e53b-5f36-4bc3-a1b0-fbb5a367687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:10.383820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e53b-5f36-4bc3-a1b0-fbb5a367687b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.384293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e53b-5f36-4bc3-a1b0-fbb5a367687b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.385156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e53b-5f36-4bc3-a1b0-fbb5a367687b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.386686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e452f-78f2-4b7f-8c00-b186eb6734a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:10.388211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e452f-78f2-4b7f-8c00-b186eb6734a8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.388721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e452f-78f2-4b7f-8c00-b186eb6734a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.389199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e452f-78f2-4b7f-8c00-b186eb6734a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.395898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e53b-5f36-4bc3-a1b0-fbb5a367687b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:10.401881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e452f-78f2-4b7f-8c00-b186eb6734a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-d9a5e53b-5f36-4bc3-a1b0-fbb5a367687b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:10.408060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:10.409087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:10.409855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-de5e452f-78f2-4b7f-8c00-b186eb6734a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:10.410429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:10.411783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 33/66] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:10.412793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0af09-793d-4c35-8866-9e4db499669f None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.413496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0af09-793d-4c35-8866-9e4db499669f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.413496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0af09-793d-4c35-8866-9e4db499669f None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.414100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 34/67] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:10.415588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 34/68] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:10.418887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0af09-793d-4c35-8866-9e4db499669f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.419631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc921-d4e9-4cf5-b768-977a8a156af3 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.420396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc921-d4e9-4cf5-b768-977a8a156af3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.420906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc921-d4e9-4cf5-b768-977a8a156af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.437915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-38bbc921-d4e9-4cf5-b768-977a8a156af3 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:10.439460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-38bbc921-d4e9-4cf5-b768-977a8a156af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:10.476944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:10.516656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 35/69] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:10.516656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbc921-d4e9-4cf5-b768-977a8a156af3 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.541714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0af09-793d-4c35-8866-9e4db499669f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:10.557727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:10.575767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d641b-ad18-4e54-9105-a016dc4dc54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:10.576457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d641b-ad18-4e54-9105-a016dc4dc54e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.576757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d641b-ad18-4e54-9105-a016dc4dc54e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.577042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d641b-ad18-4e54-9105-a016dc4dc54e None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.577796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d641b-ad18-4e54-9105-a016dc4dc54e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:10.578110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d641b-ad18-4e54-9105-a016dc4dc54e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:10.579502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d641b-ad18-4e54-9105-a016dc4dc54e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:10.590353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 35/70] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:10.591659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b32b4f-a141-4621-b526-131a3ae509a4 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.592448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b32b4f-a141-4621-b526-131a3ae509a4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.592745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b32b4f-a141-4621-b526-131a3ae509a4 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.594151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 36/71] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:10.594726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b32b4f-a141-4621-b526-131a3ae509a4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.619632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbc921-d4e9-4cf5-b768-977a8a156af3 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:10.657002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30ffa3-613e-41a1-b99f-2a636d971c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:10.657694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30ffa3-613e-41a1-b99f-2a636d971c30 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.657987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30ffa3-613e-41a1-b99f-2a636d971c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.658276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30ffa3-613e-41a1-b99f-2a636d971c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.659042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30ffa3-613e-41a1-b99f-2a636d971c30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:10.659358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30ffa3-613e-41a1-b99f-2a636d971c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:10.660786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30ffa3-613e-41a1-b99f-2a636d971c30 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:10.669481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 36/72] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:10.670780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7808e54-8b82-452e-be79-360aa1f6d397 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.671079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7808e54-8b82-452e-be79-360aa1f6d397 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.671364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7808e54-8b82-452e-be79-360aa1f6d397 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.672701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 37/73] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:10.673366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7808e54-8b82-452e-be79-360aa1f6d397 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.699952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b32b4f-a141-4621-b526-131a3ae509a4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:10.708083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:10.734560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f771-e2cf-4f7f-92e3-4b5c3829d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:10.735374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f771-e2cf-4f7f-92e3-4b5c3829d2da None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.735669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f771-e2cf-4f7f-92e3-4b5c3829d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.735983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f771-e2cf-4f7f-92e3-4b5c3829d2da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.746581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f771-e2cf-4f7f-92e3-4b5c3829d2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-8250f771-e2cf-4f7f-92e3-4b5c3829d2da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:10.754488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:10.755544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:10.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:10.757617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70207a-23d1-457e-9dc0-8fd50544fb29 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.757617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70207a-23d1-457e-9dc0-8fd50544fb29 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.757617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70207a-23d1-457e-9dc0-8fd50544fb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.773040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-dd70207a-23d1-457e-9dc0-8fd50544fb29 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:10.774176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-dd70207a-23d1-457e-9dc0-8fd50544fb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:10.778059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7808e54-8b82-452e-be79-360aa1f6d397 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:10.803620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:10.811495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:10.812372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.812688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.816053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] PATH_INFO: `/v3/users/85af2924b4aa4547ad22ecda72bbb229` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.816053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] RBAC: Authorizing `identity:get_user(user_id=85af2924b4aa4547ad22ecda72bbb229)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:10.816053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:10.817392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/users/85af2924b4aa4547ad22ecda72bbb229 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:10.818156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.841075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 38/76] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:10.842328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42493d1f-16dc-45a6-b90c-d4ca0917a4e8 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.842619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42493d1f-16dc-45a6-b90c-d4ca0917a4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.842900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42493d1f-16dc-45a6-b90c-d4ca0917a4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.858490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-42493d1f-16dc-45a6-b90c-d4ca0917a4e8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:10.859701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-42493d1f-16dc-45a6-b90c-d4ca0917a4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:10.918867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51a9dc-6d55-4aa1-bbe9-5395b5beea8e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:10.920360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:10.926934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 39/77] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:10.927775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42493d1f-16dc-45a6-b90c-d4ca0917a4e8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:10.956342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:10.957249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:10.957541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:10.957823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:10.971637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:10.979953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:10.979953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:10.979953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.034391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.038078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42493d1f-16dc-45a6-b90c-d4ca0917a4e8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.072063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.072772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.073181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.073431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.074147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:11.074454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.075484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b4ab1-a408-4c64-892b-1017201e60f3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.075842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.083638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 40/79] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:11.084414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.120749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.121413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.121705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.121987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.149805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.185598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca68ae-bd64-40ee-aa95-ba1fda0e6d8c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.191440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.192977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.218478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.219262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.219546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.219822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.221110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.222553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.224187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 41/80] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:11.224978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.254563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.291483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 40/81] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:11.295420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.325406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb70533-5b4d-4355-bc74-b682bdc9f6a7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.350805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.358167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.361859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.361859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.361859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.369855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:11.377552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:11.378412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 42/82] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:11.384239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.391693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab464d5-e742-4198-b60e-e56ba81cf15e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.425678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.426342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.426634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.426915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.467619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.479610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba9e6e-af4e-4026-bc87-874aeedb6211 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.494884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.496344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.512706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.513581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.513876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.514159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.515383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.516874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.518516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 43/83] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:11.519269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.552501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.592596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 41/84] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:11.593420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.625369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bbef7-49d4-4ad3-a2ed-357a0a9c1b64 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.658444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.659115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.659467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.659685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.660414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.661897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.692992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7b9d5-68be-406b-bdae-d4119ec60ebe None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.726378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.727237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.727527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.727814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.729571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.731030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.732762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 42/85] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:11.733527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.761111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.772887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 44/86] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:29:11 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:11.773716 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.791892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:11.842685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6f93d-0594-4219-9073-da7c1c8eb16f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.881704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.882695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.883496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.884206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:11.885704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:11.886169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:11.886631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ccb3a-e2ab-4cde-b916-34a33ae43535 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:11.887949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:11.901094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 43/87] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:11.902194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:11.936890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:11.937765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:11.938217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:11.938636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:12.053089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:12.063157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95add1a-837e-4a96-911a-01011d4a5b6d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:12.065970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:12.067488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:12.082324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:12.110386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:12.111274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:12.111564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:12.111845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:12.113116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:12.114589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:12.116774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 44/88] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:12.117549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:12.189339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 45/89] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:12.190423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:12.258700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4834-3265-4a08-9970-80bb628c749b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:12.300353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e5996-6e59-4b03-8777-693bfc87f229 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:12.305451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:12.306175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:12.307005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:12.307284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:12.308128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=85af2924b4aa4547ad22ecda72bbb229) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:12.308460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:12.309990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:12.317803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 45/90] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:12 2022] GET /identity/v3/role_assignments?user.id=85af2924b4aa4547ad22ecda72bbb229&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:12.321008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:12.326764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:12.366853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:12.367662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:12.367948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:12.368227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:12.371177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:12.371478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:12.373021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:12.374922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 46/91] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:12 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:12.375531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:12.537640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4170e-97e1-4f0e-92d6-9a40a70aea21 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:12.588124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1afde8-3fe0-4909-8532-0a96506fcd8f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:12.606950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:12.607677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:12.607965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:12.608246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:12.609165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=743bab5749aa46e6a980c8dede62afdd) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:12.609489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:12.614036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:12.626409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 46/92] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:12 2022] GET /identity/v3/role_assignments?user.id=743bab5749aa46e6a980c8dede62afdd&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:12.626987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:12.663255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:12.664159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:12.664465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:12.664756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:12.674621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:12.676137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:12.680143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 47/93] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:12 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:12.685889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:12.801060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:12.830769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:12.865617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a8e7-167f-44bd-be37-f3640c3a453a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:12.874432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b92cc7-6881-4c08-8ed1-77027f662f42 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:12.929571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:12.930246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:12.930536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:12.930822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:12.931735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] RBAC: Adding query filter params (role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=743bab5749aa46e6a980c8dede62afdd) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:12.932052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:12.933717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:12.939862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 48/94] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:12 2022] GET /identity/v3/role_assignments?user.id=743bab5749aa46e6a980c8dede62afdd&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=4182f79f5cda459497c049f3f07a1713 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:12.941507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:12.942190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:12.942485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:12.942769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:13.060406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:13.061926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:13.196903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:13.203174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 47/95] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:13.208909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:13.338126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4767dc-5037-45a4-af0b-a7461fba242f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:13.356851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:13.380057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64316e69-1583-4419-ba67-6f679a00c738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:13.380980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316e69-1583-4419-ba67-6f679a00c738 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:13.381275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316e69-1583-4419-ba67-6f679a00c738 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:13.381555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316e69-1583-4419-ba67-6f679a00c738 None admin] PATH_INFO: `/v3/projects/416afd7bbb6747c096ad45d200c8e68d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:13.383397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316e69-1583-4419-ba67-6f679a00c738 None admin] RBAC: Authorizing `identity:get_project(project_id=416afd7bbb6747c096ad45d200c8e68d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:13.385969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316e69-1583-4419-ba67-6f679a00c738 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:13.389611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 49/96] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:13 2022] GET /identity/v3/projects/416afd7bbb6747c096ad45d200c8e68d => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:13.398473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:13.513732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0c91-ce21-4512-bb1f-b0eb5fea2c0a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:13.553929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba9899-c1fe-4731-99e1-c9e234d3d498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:13.554601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba9899-c1fe-4731-99e1-c9e234d3d498 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:13.554888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba9899-c1fe-4731-99e1-c9e234d3d498 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:13.555168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba9899-c1fe-4731-99e1-c9e234d3d498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:13.556015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba9899-c1fe-4731-99e1-c9e234d3d498 None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=416afd7bbb6747c096ad45d200c8e68d, user.id=743bab5749aa46e6a980c8dede62afdd) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:13.556362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba9899-c1fe-4731-99e1-c9e234d3d498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:13.557866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba9899-c1fe-4731-99e1-c9e234d3d498 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:13.563699 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 48/97] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:13 2022] GET /identity/v3/role_assignments?user.id=743bab5749aa46e6a980c8dede62afdd&scope.project.id=416afd7bbb6747c096ad45d200c8e68d&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:14.361208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5f50d-bc84-4f79-8425-d68ffa6c8ce2 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:14.361697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5f50d-bc84-4f79-8425-d68ffa6c8ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:14.361697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5f50d-bc84-4f79-8425-d68ffa6c8ce2 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:14.363238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 50/98] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:14.368192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e6836-9cc1-46af-8696-357d0903b190 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:14.368596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e6836-9cc1-46af-8696-357d0903b190 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:14.368839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e6836-9cc1-46af-8696-357d0903b190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:14.390442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c57e6836-9cc1-46af-8696-357d0903b190 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:14.391764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c57e6836-9cc1-46af-8696-357d0903b190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:14.454692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:14.483258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 49/99] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:14.522241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5f50d-bc84-4f79-8425-d68ffa6c8ce2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:14.641866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5f50d-bc84-4f79-8425-d68ffa6c8ce2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:14.685921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:14.686728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:14.687063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:14.687348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:14.697894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:14.710142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:14.711026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:14.711782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 51/100] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:14.718003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e6836-9cc1-46af-8696-357d0903b190 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:14.823462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e6836-9cc1-46af-8696-357d0903b190 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:14.863201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:14.864287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:14.864785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:14.865254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:14.866299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:14.866799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:14.868363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:14.880132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 50/101] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:14.886200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:14.954178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:15.029523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be8575-e5df-491f-a899-36196555eb7a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:15.075923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3af73-4c0b-4a64-b9dc-6cf64d4ee1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:15.076851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3af73-4c0b-4a64-b9dc-6cf64d4ee1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.077143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3af73-4c0b-4a64-b9dc-6cf64d4ee1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.077426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3af73-4c0b-4a64-b9dc-6cf64d4ee1f4 None admin] PATH_INFO: `/v3/users/fa51eba5aae74797908d267434454354` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.099450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3af73-4c0b-4a64-b9dc-6cf64d4ee1f4 None admin] RBAC: Authorizing `identity:get_user(user_id=fa51eba5aae74797908d267434454354)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:15.099784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3af73-4c0b-4a64-b9dc-6cf64d4ee1f4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:15.101765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 52/102] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:14 2022] GET /identity/v3/users/fa51eba5aae74797908d267434454354 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:15.107997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:15.162480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1fee9-2898-4f4d-b686-a96f89827345 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.162780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1fee9-2898-4f4d-b686-a96f89827345 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.163340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1fee9-2898-4f4d-b686-a96f89827345 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.165211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 53/103] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:15.174517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee977f-4ff9-4249-aa81-7209a0807958 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.174812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee977f-4ff9-4249-aa81-7209a0807958 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.175516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee977f-4ff9-4249-aa81-7209a0807958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.203318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5bee977f-4ff9-4249-aa81-7209a0807958 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:15.204632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5bee977f-4ff9-4249-aa81-7209a0807958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:15.206340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:15.335382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 54/104] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:15.355069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46d3e-4057-4d45-aec5-4613bb3fd368 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.355366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46d3e-4057-4d45-aec5-4613bb3fd368 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.355911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46d3e-4057-4d45-aec5-4613bb3fd368 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.357321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 55/105] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:15.362496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426dd07-ff62-49d5-a534-53aa8f434603 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.363038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426dd07-ff62-49d5-a534-53aa8f434603 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.363038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426dd07-ff62-49d5-a534-53aa8f434603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.368195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b0095-a1bc-4e81-b486-2dd162df77b5 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:15.383932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c426dd07-ff62-49d5-a534-53aa8f434603 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:15.385292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c426dd07-ff62-49d5-a534-53aa8f434603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:15.410377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:15.420093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669b53d-c1ab-4e69-bb12-047699107f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:15.420786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669b53d-c1ab-4e69-bb12-047699107f84 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.421077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669b53d-c1ab-4e69-bb12-047699107f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.421716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669b53d-c1ab-4e69-bb12-047699107f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.457372 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 56/106] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:15.458243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426dd07-ff62-49d5-a534-53aa8f434603 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:15.501186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669b53d-c1ab-4e69-bb12-047699107f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:15.502672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669b53d-c1ab-4e69-bb12-047699107f84 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:15.551899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:15.588884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426dd07-ff62-49d5-a534-53aa8f434603 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:15.614701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 51/107] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:15.616050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95286f-c108-4bc8-bce9-05824d3f99d6 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.616344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95286f-c108-4bc8-bce9-05824d3f99d6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.616651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95286f-c108-4bc8-bce9-05824d3f99d6 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.617945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 52/108] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:15.618638 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95286f-c108-4bc8-bce9-05824d3f99d6 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:15.637103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:15.637905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.638294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.638579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.639862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:15.641519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:15.643276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 57/109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:15.644111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:15.674194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:15.734467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95286f-c108-4bc8-bce9-05824d3f99d6 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:15.761754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1780e-d483-48b2-a035-7af0ee28afe7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:15.771555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf929f2-701c-450e-916b-e27f10d9b1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:15.772499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf929f2-701c-450e-916b-e27f10d9b1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.772795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf929f2-701c-450e-916b-e27f10d9b1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.773080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf929f2-701c-450e-916b-e27f10d9b1b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.785145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf929f2-701c-450e-916b-e27f10d9b1b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-aaf929f2-701c-450e-916b-e27f10d9b1b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:15.793387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:15.795422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 53/110] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:15.797409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32386d-ff9b-4f77-98ba-07788ed54e23 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.799093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32386d-ff9b-4f77-98ba-07788ed54e23 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.799093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32386d-ff9b-4f77-98ba-07788ed54e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.809140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:15.809404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:15.810026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.810559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.810888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.813065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:15.814679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:15.816563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 58/111] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:15 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:15.817311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:15.825437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8b32386d-ff9b-4f77-98ba-07788ed54e23 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:15.826662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8b32386d-ff9b-4f77-98ba-07788ed54e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:15.897529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 54/112] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:15.898506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32386d-ff9b-4f77-98ba-07788ed54e23 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:15.924981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:15.932820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345acc7-3009-4204-9a31-95504a31a49f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:15.970719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:15.971496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:15.971791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:15.972076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:15.972845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:15.974429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:16.007459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32386d-ff9b-4f77-98ba-07788ed54e23 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.037950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:16.043070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.043768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.044061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.044346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.045169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:16.045620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:16.046884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:16.058894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 55/113] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:16.060048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.079884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 59/114] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:29:15 2022] POST /identity/v3/projects => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:16.081571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.240705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3808d9-a6b0-4d32-b1e8-4fd862650229 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.269425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3680a-b6ab-49e9-b5a2-0665ac8bcb49 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.287999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.288704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.288996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.289280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.290130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] RBAC: Adding query filter params (role.id=a3a50df9b07a42c4b1361e06ea200403, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=fa51eba5aae74797908d267434454354) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:16.291185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:16.295176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:16.304215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 56/115] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/role_assignments?user.id=fa51eba5aae74797908d267434454354&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=a3a50df9b07a42c4b1361e06ea200403 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:16.304945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.314571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543097b6-f5f0-4592-9bf3-e01cb4b92080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.315750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543097b6-f5f0-4592-9bf3-e01cb4b92080 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.316224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543097b6-f5f0-4592-9bf3-e01cb4b92080 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.316719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543097b6-f5f0-4592-9bf3-e01cb4b92080 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.332821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543097b6-f5f0-4592-9bf3-e01cb4b92080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-543097b6-f5f0-4592-9bf3-e01cb4b92080 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:16.344312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:16.345260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:16.346026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:16.349127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 60/116] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:16.350727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f92a2-6889-43e9-a766-0a099f144c25 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.351327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f92a2-6889-43e9-a766-0a099f144c25 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.351766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f92a2-6889-43e9-a766-0a099f144c25 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.367323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 61/117] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:16.369099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f92a2-6889-43e9-a766-0a099f144c25 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.380918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:16.389394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:16.439558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c6845-6f7a-408d-8e34-5daeb620fff9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.480635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a079d4-98ec-4d3d-8008-3ab0bac286aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.481815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a079d4-98ec-4d3d-8008-3ab0bac286aa None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.482251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a079d4-98ec-4d3d-8008-3ab0bac286aa None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.482661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a079d4-98ec-4d3d-8008-3ab0bac286aa None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.484137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a079d4-98ec-4d3d-8008-3ab0bac286aa None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:16.485931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a079d4-98ec-4d3d-8008-3ab0bac286aa None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:16.487789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 57/118] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:16.489167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6018778-ab3d-4dc6-a417-8fe7f7b5fe1d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.489990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6018778-ab3d-4dc6-a417-8fe7f7b5fe1d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.490432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6018778-ab3d-4dc6-a417-8fe7f7b5fe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.493735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f92a2-6889-43e9-a766-0a099f144c25 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.514819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f6018778-ab3d-4dc6-a417-8fe7f7b5fe1d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:16.516523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f6018778-ab3d-4dc6-a417-8fe7f7b5fe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:16.561283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.562231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.562695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.563211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.564229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:16.564740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:16.566344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:16.576886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 62/119] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:16.577989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.599731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 58/120] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:16.605805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e77e-3ffc-45ce-a81e-e7c62065d47e None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.606305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e77e-3ffc-45ce-a81e-e7c62065d47e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.606804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e77e-3ffc-45ce-a81e-e7c62065d47e None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.608322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 59/121] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:16.610420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485e77e-3ffc-45ce-a81e-e7c62065d47e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.669031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:16.688808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:16.704056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcac52-20ef-41a7-b604-017dddb4669e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.729934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485e77e-3ffc-45ce-a81e-e7c62065d47e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.739959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c518c6-cc8f-4bec-b572-c6c18252a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.740874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c518c6-cc8f-4bec-b572-c6c18252a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.741210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c518c6-cc8f-4bec-b572-c6c18252a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.741499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c518c6-cc8f-4bec-b572-c6c18252a7b2 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.742720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c518c6-cc8f-4bec-b572-c6c18252a7b2 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:16.744257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c518c6-cc8f-4bec-b572-c6c18252a7b2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:16.746121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 63/122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:16.752041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452526-2d60-49a6-9085-3c70d9a0a121 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.752343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452526-2d60-49a6-9085-3c70d9a0a121 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.752669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13452526-2d60-49a6-9085-3c70d9a0a121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.768836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.770080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.770573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.771009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-13452526-2d60-49a6-9085-3c70d9a0a121 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:16.771262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] PATH_INFO: `/v3/users/85af2924b4aa4547ad22ecda72bbb229` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.772481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-13452526-2d60-49a6-9085-3c70d9a0a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:16.772919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] RBAC: Authorizing `identity:get_user(user_id=85af2924b4aa4547ad22ecda72bbb229)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:16.774694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:16.777026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 60/123] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/users/85af2924b4aa4547ad22ecda72bbb229 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:16.778488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.856839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 64/124] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:16.857795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13452526-2d60-49a6-9085-3c70d9a0a121 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.896669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c825e0-8ec3-491a-978a-3541b7666106 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:16.926681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:16.938641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:16.939693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:16.940199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:16.940703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:16.942558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:16.952315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:16.961079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:16.962002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:16.963303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 61/125] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:16.969101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:16.988888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13452526-2d60-49a6-9085-3c70d9a0a121 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:17.070857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:17.072864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:17.073254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:17.073636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/743bab5749aa46e6a980c8dede62afdd/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:17.077013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a43fae56142df9ddb88b6d5c798fb, user_id=743bab5749aa46e6a980c8dede62afdd, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:17.078590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:17.110190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2fdc4-ffbd-44d5-859d-31712cb7bb84 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:17.145714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:17.146927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:17.147102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:17.147349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:17.148847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:17.150407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:17.152212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 62/126] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:16 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:17.152601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 65/127] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:16 2022] PUT /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/743bab5749aa46e6a980c8dede62afdd/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:17.153650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:17.158554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:17.264988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:17.280490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c70b04-bb24-41c0-b8f2-e8fe342b2353 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:17.314634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:17.326810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:17.328743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:17.329186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:17.329595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:17.330509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:17.330951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:17.332574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:17.344313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 66/128] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:17.345535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:17.358375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67172d-aa30-4324-8be4-6cf3ce51014e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:17.438216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:17.439713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:17.440661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:17.441086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:17.455993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:17.466916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:17.468000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:17.468901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:17.470094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 63/129] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:17.470600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:17.474938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c18989-c244-43b9-b108-ec107a3020d6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:17.512667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:17.513571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:17.513862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:17.514144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/85af2924b4aa4547ad22ecda72bbb229/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:17.516895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a43fae56142df9ddb88b6d5c798fb, user_id=85af2924b4aa4547ad22ecda72bbb229, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:17.518458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:17.589830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c1c3-5749-4716-9669-3d165052f74c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:17.611418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:17.629619 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:17.630167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 67/130] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:17 2022] PUT /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/85af2924b4aa4547ad22ecda72bbb229/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:17.631202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:17.637130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:17.637878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:17.639892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:17.642506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:17.646325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:17.649745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 64/131] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:17 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:17.657895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:17.769035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:17.811119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06c48d-7d48-44c7-8c42-bf1b3547efdf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:17.873343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:17.874037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:17.874327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:17.874606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:17.875377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:17.875690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:17.877114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:17.892066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 68/132] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:17.897440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:17.951220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585f07b-d76a-4c02-9ef6-7cbd0e9ecd41 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:18.011570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:18.012854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.013245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.013626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.014967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:18.016570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:18.018843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 65/133] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:17 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:18.023239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4baf93-3b05-4082-b013-f60e7649b6c3 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:18.030188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:18.046804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:18.103772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:18.104696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.104986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.105267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.106656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:18.108121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:18.109974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 69/134] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:17 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:18.123557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:18.161969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:18.293330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894d9b4-e07c-4b1b-bfd2-9788c798c8c1 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:18.330375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf568ee9-0f48-4c8a-8b73-b93032610f2b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:18.362121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea3969-d929-448a-8e5f-ba17692ee2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:18.363040 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3969-d929-448a-8e5f-ba17692ee2e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.363334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3969-d929-448a-8e5f-ba17692ee2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.363619 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3969-d929-448a-8e5f-ba17692ee2e1 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/743bab5749aa46e6a980c8dede62afdd/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.370874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea3969-d929-448a-8e5f-ba17692ee2e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a43fae56142df9ddb88b6d5c798fb, user_id=743bab5749aa46e6a980c8dede62afdd, role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:18.372352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea3969-d929-448a-8e5f-ba17692ee2e1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:18.413205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:18.414099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.414393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.414681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] PATH_INFO: `/v3/projects/416afd7bbb6747c096ad45d200c8e68d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.420576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] RBAC: Authorizing `identity:get_project(project_id=416afd7bbb6747c096ad45d200c8e68d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:18.422124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:18.423962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 70/135] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:18 2022] GET /identity/v3/projects/416afd7bbb6747c096ad45d200c8e68d => generated 312 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:18.430479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:18.476795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 66/136] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:18 2022] PUT /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/743bab5749aa46e6a980c8dede62afdd/roles/4182f79f5cda459497c049f3f07a1713 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:18.487065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:18.568006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f340c-2d0f-4870-8ad3-0ae440a38b1d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:18.578116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80d10-04d8-49c7-b33b-15d9293de80e None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.578674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80d10-04d8-49c7-b33b-15d9293de80e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.579125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80d10-04d8-49c7-b33b-15d9293de80e None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.580688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 67/137] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:18.601962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f8d0a-4859-44d9-b087-212557007c16 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.602555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f8d0a-4859-44d9-b087-212557007c16 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.603122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f8d0a-4859-44d9-b087-212557007c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.610031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:18.611033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.611333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.611623 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] PATH_INFO: `/v3/projects/416afd7bbb6747c096ad45d200c8e68d/users/743bab5749aa46e6a980c8dede62afdd/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.616459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] RBAC: Authorizing `identity:create_grant(project_id=416afd7bbb6747c096ad45d200c8e68d, user_id=743bab5749aa46e6a980c8dede62afdd, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:18.616459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:18.631963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-231f8d0a-4859-44d9-b087-212557007c16 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:18.633412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-231f8d0a-4859-44d9-b087-212557007c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:18.703507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 71/138] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:18 2022] PUT /identity/v3/projects/416afd7bbb6747c096ad45d200c8e68d/users/743bab5749aa46e6a980c8dede62afdd/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:29:18.731079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 68/139] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:18.770055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:18.879944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a038db-e60f-484e-b1b0-c346f9707097 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.880588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a038db-e60f-484e-b1b0-c346f9707097 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.881067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a038db-e60f-484e-b1b0-c346f9707097 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.882633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 69/140] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:18.889934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099720-9c7b-409e-92f7-77b5e2a1c5b2 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.890226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099720-9c7b-409e-92f7-77b5e2a1c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.890511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099720-9c7b-409e-92f7-77b5e2a1c5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.900242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae1432-782a-41f0-8ee1-437b26d64e6f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:18.912188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:18.919951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-77099720-9c7b-409e-92f7-77b5e2a1c5b2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:18.921192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-77099720-9c7b-409e-92f7-77b5e2a1c5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:18.950163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:18.951197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:18.951492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:18.952158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:18.960896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:18.969280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:18.980158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:18.982266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:18.983114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 72/141] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:18.987455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:19.006747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 70/142] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:19.032954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77099720-9c7b-409e-92f7-77b5e2a1c5b2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:19.109576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6441b5-dddf-49b3-a25c-d318600960ff None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:19.146597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:19.147291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:19.147581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:19.147859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:19.148800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:19.149158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:19.151138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:19.160504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 73/143] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:19.165513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77099720-9c7b-409e-92f7-77b5e2a1c5b2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:19.166403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:19.204573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:19.234917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:19.235784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:19.236076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:19.236350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:19.239094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:19.241644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:19.249548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 71/144] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:19.264240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:19.344565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd07ce1-869f-4fe7-b4c8-9e173e9937d0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:19.351493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:19.390177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:19.391069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:19.391356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:19.391638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] PATH_INFO: `/v3/users/fa51eba5aae74797908d267434454354` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:19.392904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] RBAC: Authorizing `identity:get_user(user_id=fa51eba5aae74797908d267434454354)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:19.394555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:19.396549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 74/145] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:19 2022] GET /identity/v3/users/fa51eba5aae74797908d267434454354 => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:19.407745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:19.514631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd7348-5b6a-4d88-b70b-810bfa38757f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:19.560820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:19.566429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65129242-a852-43ae-9fab-08e95f73d7ad None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:19.609682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65f7fc-5f92-40ae-92af-fc3dd1989c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:19.610364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f7fc-5f92-40ae-92af-fc3dd1989c1a None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:19.610648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f7fc-5f92-40ae-92af-fc3dd1989c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:19.611035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f7fc-5f92-40ae-92af-fc3dd1989c1a None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:19.611717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65f7fc-5f92-40ae-92af-fc3dd1989c1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:19.612802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:19.613311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65f7fc-5f92-40ae-92af-fc3dd1989c1a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:19.613687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:19.613983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:19.614273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:19.615509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:19.617370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:19.619813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 75/146] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:19 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:19.632307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:19.748366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 72/147] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:29:19 2022] POST /identity/v3/users => generated 283 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:29:19.795522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcaa26-ad1d-460d-9d5a-539f6d5c7d31 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:19.843219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307ab18-e8e2-4fa1-8f18-9d5bf259313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:19.844585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ab18-e8e2-4fa1-8f18-9d5bf259313a None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:19.845094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ab18-e8e2-4fa1-8f18-9d5bf259313a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:19.845585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ab18-e8e2-4fa1-8f18-9d5bf259313a None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/fa51eba5aae74797908d267434454354/roles/a3a50df9b07a42c4b1361e06ea200403` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:19.848013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307ab18-e8e2-4fa1-8f18-9d5bf259313a None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a43fae56142df9ddb88b6d5c798fb, user_id=fa51eba5aae74797908d267434454354, role_id=a3a50df9b07a42c4b1361e06ea200403)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:19.849823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307ab18-e8e2-4fa1-8f18-9d5bf259313a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:19.933751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 76/148] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:19 2022] PUT /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/users/fa51eba5aae74797908d267434454354/roles/a3a50df9b07a42c4b1361e06ea200403 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:20.436716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4b39-fc21-4896-91f6-920bdfb7a705 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:20.437089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4b39-fc21-4896-91f6-920bdfb7a705 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:20.437295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4b39-fc21-4896-91f6-920bdfb7a705 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:20.438814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 73/149] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:20.444134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578f7c7-11bd-4988-bed5-239eaee42645 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:20.444722 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578f7c7-11bd-4988-bed5-239eaee42645 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:20.445221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578f7c7-11bd-4988-bed5-239eaee42645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:20.464766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e578f7c7-11bd-4988-bed5-239eaee42645 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:20.466290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e578f7c7-11bd-4988-bed5-239eaee42645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:20.485457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:20.542535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 77/150] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:20.568484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e4b39-fc21-4896-91f6-920bdfb7a705 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:20.668620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:20.679605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e4b39-fc21-4896-91f6-920bdfb7a705 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:20.718155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:20.719298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:20.719884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:20.720394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:20.732599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:20.740932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:20.741871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:20.742680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:20.745687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 74/151] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:20.753068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578f7c7-11bd-4988-bed5-239eaee42645 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:20.863044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578f7c7-11bd-4988-bed5-239eaee42645 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:20.900960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:20.902006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:20.902492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:20.902982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:20.903959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:20.904498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:20.906122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:20.916286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 78/152] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:20.922192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:21.034628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc0c12-6d67-4d10-b9ff-8b23eb087851 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:21.070743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0264c-0977-4655-886c-92d0fee911be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:21.072632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0264c-0977-4655-886c-92d0fee911be None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.073123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0264c-0977-4655-886c-92d0fee911be None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.073616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0264c-0977-4655-886c-92d0fee911be None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.075059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0264c-0977-4655-886c-92d0fee911be None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:21.076786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0264c-0977-4655-886c-92d0fee911be None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:21.078878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 75/153] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:20 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:21.084645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:21.090662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158be2a-e008-46e6-bf5c-2e439e501ad8 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.090662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158be2a-e008-46e6-bf5c-2e439e501ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.090662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158be2a-e008-46e6-bf5c-2e439e501ad8 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.091946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 76/154] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:21.107422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057099-4991-45c9-86b2-5bf2047b4ba9 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.107722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057099-4991-45c9-86b2-5bf2047b4ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.108002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057099-4991-45c9-86b2-5bf2047b4ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.113356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:21.141469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5d057099-4991-45c9-86b2-5bf2047b4ba9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:21.142888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5d057099-4991-45c9-86b2-5bf2047b4ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:21.173896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:21.219256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a704f0e-b4d1-4a6a-89d5-79551cd6ee80 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:21.258156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef3977-7974-41aa-9b06-5f91acaf524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:21.259885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef3977-7974-41aa-9b06-5f91acaf524a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.260400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef3977-7974-41aa-9b06-5f91acaf524a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.260900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef3977-7974-41aa-9b06-5f91acaf524a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.307644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 77/155] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:21.349569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef3977-7974-41aa-9b06-5f91acaf524a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:21.351409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef3977-7974-41aa-9b06-5f91acaf524a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:21.352800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057099-4991-45c9-86b2-5bf2047b4ba9 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:21.373030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:21.464952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 79/156] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:21.467170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4c41-b97c-44f8-b55b-ae6f7177f9c1 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.472816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4c41-b97c-44f8-b55b-ae6f7177f9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.473305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4c41-b97c-44f8-b55b-ae6f7177f9c1 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.474926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 80/157] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:21.475825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a4c41-b97c-44f8-b55b-ae6f7177f9c1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:21.489694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057099-4991-45c9-86b2-5bf2047b4ba9 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:21.536703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f83eb8-58f9-4cdc-b2f4-09db757db6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:21.537750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83eb8-58f9-4cdc-b2f4-09db757db6bc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.538882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83eb8-58f9-4cdc-b2f4-09db757db6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.539373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83eb8-58f9-4cdc-b2f4-09db757db6bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.562207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f83eb8-58f9-4cdc-b2f4-09db757db6bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-29f83eb8-58f9-4cdc-b2f4-09db757db6bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:21.569997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:21.571511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:21.572569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:21.575903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 78/158] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:21.577259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf489d1-b850-4d56-99f4-e76ca34c84fd None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.577554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf489d1-b850-4d56-99f4-e76ca34c84fd None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.577834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf489d1-b850-4d56-99f4-e76ca34c84fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.612608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0cf489d1-b850-4d56-99f4-e76ca34c84fd None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:21.613814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0cf489d1-b850-4d56-99f4-e76ca34c84fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:21.622252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:21.650578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a4c41-b97c-44f8-b55b-ae6f7177f9c1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:21.689184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:21.728824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 79/159] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:21.730120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf4d19-53f4-4677-8ca2-1bd37986f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:21.730575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92530d-9bf0-4793-8d7b-c7703f5fdcc3 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.730928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92530d-9bf0-4793-8d7b-c7703f5fdcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.731274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92530d-9bf0-4793-8d7b-c7703f5fdcc3 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.731628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf4d19-53f4-4677-8ca2-1bd37986f499 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.732051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf4d19-53f4-4677-8ca2-1bd37986f499 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.732477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf4d19-53f4-4677-8ca2-1bd37986f499 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.733284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 80/160] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:21.734039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf4d19-53f4-4677-8ca2-1bd37986f499 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:21.734203 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92530d-9bf0-4793-8d7b-c7703f5fdcc3 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:21.736137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf4d19-53f4-4677-8ca2-1bd37986f499 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:21.742507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 81/161] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:21 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:21.744211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4df0f-858f-44f1-8d80-664aab7bfcd9 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.744737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4df0f-858f-44f1-8d80-664aab7bfcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.745225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4df0f-858f-44f1-8d80-664aab7bfcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.774519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b3f4df0f-858f-44f1-8d80-664aab7bfcd9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:21.775864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b3f4df0f-858f-44f1-8d80-664aab7bfcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:21.894146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92530d-9bf0-4793-8d7b-c7703f5fdcc3 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:21.935886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:21.936875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:21.937317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:21.937734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:21.938643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:21.939102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:21.940825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:21.950857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 81/162] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:21.951992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:21.959169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 82/163] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:21.960092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4df0f-858f-44f1-8d80-664aab7bfcd9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:21.966401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:22.069814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8211a04-2ad3-45d0-a38e-cdf5dda39e5f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:22.134150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:22.134835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:22.135120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:22.135398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:22.136245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=743bab5749aa46e6a980c8dede62afdd) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:22.136586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:22.138048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:22.154728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 82/164] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:21 2022] GET /identity/v3/role_assignments?user.id=743bab5749aa46e6a980c8dede62afdd&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:22.158415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:22.197726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4df0f-858f-44f1-8d80-664aab7bfcd9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:22.236464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:22.272907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:22.277150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:22.278121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:22.278514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:22.278894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:22.295977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:22.316976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:22.318041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:22.318823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:22.321666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 83/165] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:22.323327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:22.334574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8373a80-d867-47ff-aed8-46e7e1e5d8bc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:22.408940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:22.409774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:22.410059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:22.410338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] PATH_INFO: `/v3/users/85af2924b4aa4547ad22ecda72bbb229` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:22.415863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] RBAC: Authorizing `identity:get_user(user_id=85af2924b4aa4547ad22ecda72bbb229)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:22.417292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:22.419082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 83/166] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:22 2022] GET /identity/v3/users/85af2924b4aa4547ad22ecda72bbb229 => generated 247 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:22.419854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:22.495585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c81c8-b0c1-48c8-b59f-7b005e440dc6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:22.566195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:22.567406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:22.567916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:22.568391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:22.590076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:22.598370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:22.599263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:22.600052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:22.605298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2973c1-cef8-49a7-93f9-ce3c03a372a9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:22.607144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 84/167] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:22.608152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:22.650650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:22.651492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:22.651633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:22.651911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:22.652772 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:22.653028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:22.654494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:22.655985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:22.660763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:22.671604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 84/168] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:22.672616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:22.728600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8ec94-e6a7-4518-aa0d-91f8ef3e156d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:22.774195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:22.775276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:22.776195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:22.776656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:22.802823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4322e-004b-4721-b373-a2ba6f94f8e3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:22.857985 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:22.859905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:22.860026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:22.860780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:22.861142 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:22.861496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:22.862328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:22.862707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:22.864563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:22.885599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 85/169] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:22.886566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:22.904231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:22.976804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 85/170] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:22.978092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:22.979751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:23.042153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b52854-723f-4fac-b020-f07eef3759c4 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:23.090071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435762d5-127e-45fc-93c8-43dfc16ddc85 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:23.099343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:23.100322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.100695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.101065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.103787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:23.103907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:23.105649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 86/171] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:22 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:23.117631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:23.142592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:23.143476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.143760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.144036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] PATH_INFO: `/v3/users/743bab5749aa46e6a980c8dede62afdd` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.145284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] RBAC: Authorizing `identity:get_user(user_id=743bab5749aa46e6a980c8dede62afdd)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:23.146740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:23.148746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 86/172] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:22 2022] GET /identity/v3/users/743bab5749aa46e6a980c8dede62afdd => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:23.149614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:23.152241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:23.308306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8786-83db-4cbd-b538-3023030e5200 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:23.354018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ecfe8-2d6b-4e1c-b200-710cfdeaf885 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:23.367667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:23.368323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.368687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.369523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.422936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:23.445674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:23.447092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.447386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.447665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.448917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:23.450482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:23.452355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 87/173] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:23 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:23.455920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:23.477116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:23.479558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:23.481672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:23.636288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 87/174] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:23.637094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:23.659685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435339c2-f45e-4c46-9cbc-af8906b85c52 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:23.698206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:23.699286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.699769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.700261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.772271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:23.774281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:23.775266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:23.841307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305e54d-d037-45f1-abc0-72c7c465cce4 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:23.844750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:23.876640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0a4f0-9885-4354-9324-be28662a3e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:23.877381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0a4f0-9885-4354-9324-be28662a3e8a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.877585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0a4f0-9885-4354-9324-be28662a3e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.877861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0a4f0-9885-4354-9324-be28662a3e8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.878780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0a4f0-9885-4354-9324-be28662a3e8a None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=85af2924b4aa4547ad22ecda72bbb229) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:23.879167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0a4f0-9885-4354-9324-be28662a3e8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:23.880631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0a4f0-9885-4354-9324-be28662a3e8a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:23.897458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 88/175] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:23 2022] GET /identity/v3/role_assignments?user.id=85af2924b4aa4547ad22ecda72bbb229&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:23.897626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 88/176] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:23.897626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:23.897758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca501e5-83e1-4a3d-b79b-024080999fd7 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.897758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca501e5-83e1-4a3d-b79b-024080999fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.897758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca501e5-83e1-4a3d-b79b-024080999fd7 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.898067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 89/177] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:23.902265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630d732-ac06-4918-bcf8-a6f41ef6e160 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:23.902601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630d732-ac06-4918-bcf8-a6f41ef6e160 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:23.902881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630d732-ac06-4918-bcf8-a6f41ef6e160 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:23.904310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 90/178] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:23.905128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630d732-ac06-4918-bcf8-a6f41ef6e160 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.001838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:24.010224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fdb71-7d9f-46e3-98cf-aa96cce1e58d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.049076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a8036-42e1-4a7f-8071-c6b6abe9560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.050354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8036-42e1-4a7f-8071-c6b6abe9560e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.050835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8036-42e1-4a7f-8071-c6b6abe9560e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.051303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a8036-42e1-4a7f-8071-c6b6abe9560e None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.053020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a8036-42e1-4a7f-8071-c6b6abe9560e None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:24.054850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a8036-42e1-4a7f-8071-c6b6abe9560e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:24.057114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 89/179] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:23 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:24.058626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b8c0-2b2b-412b-bd76-11c1a98842d7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.059140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b8c0-2b2b-412b-bd76-11c1a98842d7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.059700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90b8c0-2b2b-412b-bd76-11c1a98842d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.078293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8d90b8c0-2b2b-412b-bd76-11c1a98842d7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:24.079872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8d90b8c0-2b2b-412b-bd76-11c1a98842d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:24.133224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630d732-ac06-4918-bcf8-a6f41ef6e160 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.169736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 90/180] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:24.171585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2752de-828c-4c77-8634-0da04c91bc02 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.172231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2752de-828c-4c77-8634-0da04c91bc02 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.172738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2752de-828c-4c77-8634-0da04c91bc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.172853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.173610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.173896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.174170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] PATH_INFO: `/v3/projects/416afd7bbb6747c096ad45d200c8e68d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.175371 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] RBAC: Authorizing `identity:get_project(project_id=416afd7bbb6747c096ad45d200c8e68d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:24.176853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:24.179370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 91/181] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:23 2022] GET /identity/v3/projects/416afd7bbb6747c096ad45d200c8e68d => generated 312 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:24.180145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.192970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5e2752de-828c-4c77-8634-0da04c91bc02 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:24.194519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5e2752de-828c-4c77-8634-0da04c91bc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:24.224940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:24.268471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 91/182] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:24.269692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2752de-828c-4c77-8634-0da04c91bc02 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.295436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:24.314700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51691fcb-f678-4788-91a3-7fb8d488d407 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.354669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.355783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.356266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.356856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.357610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] RBAC: Adding query filter params (role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=743bab5749aa46e6a980c8dede62afdd) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:24.357983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:24.359445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:24.365868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 92/183] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/role_assignments?user.id=743bab5749aa46e6a980c8dede62afdd&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=4182f79f5cda459497c049f3f07a1713 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:24.366796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.381841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2752de-828c-4c77-8634-0da04c91bc02 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.421415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.422446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.422939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.423380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.424391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=416afd7bbb6747c096ad45d200c8e68d, user.id=743bab5749aa46e6a980c8dede62afdd) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:24.424903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:24.426388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:24.433786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 92/184] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/role_assignments?user.id=743bab5749aa46e6a980c8dede62afdd&scope.project.id=416afd7bbb6747c096ad45d200c8e68d&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:24.434652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.517927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:24.526547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899eda-3f88-4134-aee0-0cb633c39b42 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.533901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:24.550394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00950e-43b2-426b-91e7-6df2776a8b6c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.578102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93eed8-0b12-40c1-aa7f-3891aeaa7f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.579635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93eed8-0b12-40c1-aa7f-3891aeaa7f6b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.580341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93eed8-0b12-40c1-aa7f-3891aeaa7f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.580874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93eed8-0b12-40c1-aa7f-3891aeaa7f6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.595839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d098e8-4a6e-4f14-a6e2-551b43ed62cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.595839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d098e8-4a6e-4f14-a6e2-551b43ed62cd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.595839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d098e8-4a6e-4f14-a6e2-551b43ed62cd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.595839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d098e8-4a6e-4f14-a6e2-551b43ed62cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.596264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93eed8-0b12-40c1-aa7f-3891aeaa7f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4f93eed8-0b12-40c1-aa7f-3891aeaa7f6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:24.604048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:24.605037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:24.606169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 93/185] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:24.607032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3827a42-4b11-4736-b4b0-a9bba06fd6eb None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.607247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3827a42-4b11-4736-b4b0-a9bba06fd6eb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.607591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3827a42-4b11-4736-b4b0-a9bba06fd6eb None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.609026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 94/186] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:24.614343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3827a42-4b11-4736-b4b0-a9bba06fd6eb None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.617071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d098e8-4a6e-4f14-a6e2-551b43ed62cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-d6d098e8-4a6e-4f14-a6e2-551b43ed62cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:24.626267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:24.627298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:24.628074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:24.631078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 93/187] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:24.632832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665cde-7b17-44fb-a5d9-5779220f6b46 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.633342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665cde-7b17-44fb-a5d9-5779220f6b46 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.633817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665cde-7b17-44fb-a5d9-5779220f6b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.652006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-6b665cde-7b17-44fb-a5d9-5779220f6b46 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:24.653561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6b665cde-7b17-44fb-a5d9-5779220f6b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:24.757481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3827a42-4b11-4736-b4b0-a9bba06fd6eb None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.757777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 94/188] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:24.759989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b665cde-7b17-44fb-a5d9-5779220f6b46 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.804010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.804831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.805124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.805407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.806230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:24.806547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:24.807936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:24.819142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 95/189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:24.819787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.850142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:24.859616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:24.886591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b665cde-7b17-44fb-a5d9-5779220f6b46 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.921441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.922531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.923206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.923651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.924674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:24.925076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:24.926546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:24.934826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 95/190] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:24.935808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:24.938624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac841be-f4d4-4509-acc4-13e454cce6d5 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:24.974163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93506be4-b026-4af7-bf8e-f1c6eb81293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:24.975428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93506be4-b026-4af7-bf8e-f1c6eb81293a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.975916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93506be4-b026-4af7-bf8e-f1c6eb81293a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.976302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93506be4-b026-4af7-bf8e-f1c6eb81293a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.986284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93506be4-b026-4af7-bf8e-f1c6eb81293a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-93506be4-b026-4af7-bf8e-f1c6eb81293a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:24.994557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:24.997409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 96/191] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:24.998216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8be861-1ab8-495c-a90e-01f0df4b58f1 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:24.998911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8be861-1ab8-495c-a90e-01f0df4b58f1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:24.998911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8be861-1ab8-495c-a90e-01f0df4b58f1 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:24.998911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 97/192] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:25.000490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8be861-1ab8-495c-a90e-01f0df4b58f1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.042255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7663c85-ac69-4913-8c38-1d7fb0b16319 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.074967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.075857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.076140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.076414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] PATH_INFO: `/v3/users/979c748c6e1b43b3bdb79455171dcb10` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.083939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.084398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.090340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] RBAC: Authorizing `identity:get_user(user_id=979c748c6e1b43b3bdb79455171dcb10)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.099400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.099400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 96/193] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/users/979c748c6e1b43b3bdb79455171dcb10 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:25.099400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.105457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8be861-1ab8-495c-a90e-01f0df4b58f1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.138977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b5181-41cf-47c3-a48e-a76f16de18ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.139846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b5181-41cf-47c3-a48e-a76f16de18ef None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.140135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b5181-41cf-47c3-a48e-a76f16de18ef None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.140412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b5181-41cf-47c3-a48e-a76f16de18ef None admin] PATH_INFO: `/v3/users/fa51eba5aae74797908d267434454354` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.141619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b5181-41cf-47c3-a48e-a76f16de18ef None admin] RBAC: Authorizing `identity:get_user(user_id=fa51eba5aae74797908d267434454354)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.143058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b5181-41cf-47c3-a48e-a76f16de18ef None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.144704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 98/194] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:24 2022] GET /identity/v3/users/fa51eba5aae74797908d267434454354 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:25.145894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a956b-f981-4702-bbc4-03ea71fe7e4d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.146184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a956b-f981-4702-bbc4-03ea71fe7e4d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.146462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a956b-f981-4702-bbc4-03ea71fe7e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.163127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f97a956b-f981-4702-bbc4-03ea71fe7e4d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:25.164319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f97a956b-f981-4702-bbc4-03ea71fe7e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:25.198788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923e1e7-e9fc-4f2c-a226-a78e0e6ec77a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.228895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 99/195] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:25.229862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a956b-f981-4702-bbc4-03ea71fe7e4d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.231136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.231961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.232252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.233265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.234040 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:25.234351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.235731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.250909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 97/196] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:25.250909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.307921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.308068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.328310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a956b-f981-4702-bbc4-03ea71fe7e4d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.348945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473dab60-e8fd-423c-a467-acaafcd88120 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.362162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.362814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.363105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.363388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.381704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.382348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.382635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.382912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.428356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.429845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.448213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.449710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.504588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.505303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.526382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 100/197] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:25.527193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.549364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 98/198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:25.550194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.627667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077683b-87a0-47cf-a5b8-c36967871ccc None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.646421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb7316-36e6-438d-8551-5c26c04ba15c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.661307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.662164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.662453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.662731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] PATH_INFO: `/v3/users/979c748c6e1b43b3bdb79455171dcb10` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.663923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] RBAC: Authorizing `identity:get_user(user_id=979c748c6e1b43b3bdb79455171dcb10)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.665453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.667079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 101/199] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/users/979c748c6e1b43b3bdb79455171dcb10 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:25.669266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.680987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.681785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.682067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.682340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.683509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.685000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.686620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 99/200] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:25.687371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.714919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.715437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.768105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3897e-460e-4855-ada6-e703cb8054c0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.789363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736cfe2-be35-4f6b-9b48-d28b1d318eef None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.801368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.802227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.802505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.802773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.803947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.805406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.807696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 102/201] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:25.808458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.823981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.824872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.825160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.825438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.826635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.828087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:25.829838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 100/202] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:25.830576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:25.905718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf2924-4974-49aa-92ba-3669c931ac84 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.919649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.920276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:25.926482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd947d4-6b8b-453a-a261-788f636b5d2d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:25.938770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.939410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.939697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.939969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.959698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:25.960375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:25.960692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:25.960975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:25.961694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:25.963222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:26.006889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:26.008356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:26.084714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 101/203] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:29:25 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:26.084714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:26.121935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 103/204] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:26.122827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:26.145644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:26.202811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d59e42-1b76-409f-be7d-a6f9c2f99b26 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:26.240941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:26.241708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:26.241993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:26.242273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:26.243102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=979c748c6e1b43b3bdb79455171dcb10) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:26.244078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:26.245090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2b7d-fd2b-4c27-ad87-e61b3eb10275 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:26.246697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:26.253167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 102/205] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:26 2022] GET /identity/v3/role_assignments?user.id=979c748c6e1b43b3bdb79455171dcb10&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:26.258090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:26.277884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:26.293807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:26.294054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:26.294735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:26.295473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:26.296370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] RBAC: Adding query filter params (role.id=a3a50df9b07a42c4b1361e06ea200403, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb, user.id=fa51eba5aae74797908d267434454354) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:26.296744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:26.298262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:26.305818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 104/206] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:26 2022] GET /identity/v3/role_assignments?user.id=fa51eba5aae74797908d267434454354&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=a3a50df9b07a42c4b1361e06ea200403 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:26.405844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a42254-68b4-4f47-bd24-1eeae5adec81 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:26.447186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adc32b-4149-4eb0-aed2-b09ff0c7c097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:26.448141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adc32b-4149-4eb0-aed2-b09ff0c7c097 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:26.448552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adc32b-4149-4eb0-aed2-b09ff0c7c097 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:26.448724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adc32b-4149-4eb0-aed2-b09ff0c7c097 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:26.450822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8adc32b-4149-4eb0-aed2-b09ff0c7c097 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:26.452461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8adc32b-4149-4eb0-aed2-b09ff0c7c097 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:26.454793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 103/207] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:26 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:26.460637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:26.556491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:26.581585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8958c-ec7b-4be8-971f-aa4054cd4532 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:26.621059 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88487fb7-0292-4f1f-9507-bd74ae4f815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:26.622105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88487fb7-0292-4f1f-9507-bd74ae4f815f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:26.622607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88487fb7-0292-4f1f-9507-bd74ae4f815f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:26.623090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88487fb7-0292-4f1f-9507-bd74ae4f815f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:26.624137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88487fb7-0292-4f1f-9507-bd74ae4f815f None admin] RBAC: Adding query filter params (role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=979c748c6e1b43b3bdb79455171dcb10) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:26.624692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88487fb7-0292-4f1f-9507-bd74ae4f815f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:26.626426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88487fb7-0292-4f1f-9507-bd74ae4f815f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:26.632900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 105/208] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:26 2022] GET /identity/v3/role_assignments?user.id=979c748c6e1b43b3bdb79455171dcb10&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=4182f79f5cda459497c049f3f07a1713 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:28.359998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2d8bf-126c-4863-8988-cc1a3d6246b3 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.360503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2d8bf-126c-4863-8988-cc1a3d6246b3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.360503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2d8bf-126c-4863-8988-cc1a3d6246b3 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.362159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 104/209] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:28.367604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4e930-a8ee-4caf-908f-018c5e0d55ba None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.368040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4e930-a8ee-4caf-908f-018c5e0d55ba None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.368796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4e930-a8ee-4caf-908f-018c5e0d55ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.382590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd23c-e770-4ff7-a3ed-bbc031e5a12b None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.383221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd23c-e770-4ff7-a3ed-bbc031e5a12b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.383697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd23c-e770-4ff7-a3ed-bbc031e5a12b None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.386109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 105/210] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:28.387688 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-bed4e930-a8ee-4caf-908f-018c5e0d55ba None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:28.389060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bed4e930-a8ee-4caf-908f-018c5e0d55ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:28.391879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a921c-6bac-4b74-8c92-b437d2fe4fbc None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.392393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a921c-6bac-4b74-8c92-b437d2fe4fbc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.392920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a921c-6bac-4b74-8c92-b437d2fe4fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.412891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3c9a921c-6bac-4b74-8c92-b437d2fe4fbc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:28.415906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3c9a921c-6bac-4b74-8c92-b437d2fe4fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:28.479395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 106/211] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:28.480850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c96e0-ffc3-416d-8d66-69837ec17268 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.481147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c96e0-ffc3-416d-8d66-69837ec17268 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.481433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c96e0-ffc3-416d-8d66-69837ec17268 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.482728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 107/212] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:28.489473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb022b-b279-4cf3-a5f9-e3987c55279d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.489919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb022b-b279-4cf3-a5f9-e3987c55279d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.490346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb022b-b279-4cf3-a5f9-e3987c55279d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.500757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 106/213] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:28.503614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a921c-6bac-4b74-8c92-b437d2fe4fbc None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:28.508238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-beeb022b-b279-4cf3-a5f9-e3987c55279d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:28.510022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-beeb022b-b279-4cf3-a5f9-e3987c55279d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:28.519054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:28.583832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 108/214] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:28.584610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb022b-b279-4cf3-a5f9-e3987c55279d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:28.587027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:28.617106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a921c-6bac-4b74-8c92-b437d2fe4fbc None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:28.658021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:28.658825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.659116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.659397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.673531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:28.684115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:28.685016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 107/215] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:28.690807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:28.695472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb022b-b279-4cf3-a5f9-e3987c55279d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:28.731506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:28.732314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.732610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.732887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.742982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:28.751317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:28.752177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:28.753120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 109/216] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:28.753776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:28.763410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:28.794375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d997932-c2f1-4b4f-9e9e-1aa9c484ac81 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:28.813886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:28.826257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586778df-b882-4946-9601-3c639d2b1914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:28.827054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586778df-b882-4946-9601-3c639d2b1914 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.827342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586778df-b882-4946-9601-3c639d2b1914 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.827622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586778df-b882-4946-9601-3c639d2b1914 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.828826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586778df-b882-4946-9601-3c639d2b1914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:28.830292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586778df-b882-4946-9601-3c639d2b1914 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:28.831910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 108/217] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:28.833717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8816-e3d6-4ca9-8b77-55027c0ee3c4 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.834080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8816-e3d6-4ca9-8b77-55027c0ee3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.834365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8816-e3d6-4ca9-8b77-55027c0ee3c4 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.835777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 109/218] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:28.837681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb8816-e3d6-4ca9-8b77-55027c0ee3c4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:28.852130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e241-62ca-46cd-adb0-774e918b64ae None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:28.895247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:28.895838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.896125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.896402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.897243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:28.897557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:28.898945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:28.907453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 110/219] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:28.908220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:28.939776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb8816-e3d6-4ca9-8b77-55027c0ee3c4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:28.972302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05180a-6892-4202-8f01-138449bdce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:28.973513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05180a-6892-4202-8f01-138449bdce07 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.973513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05180a-6892-4202-8f01-138449bdce07 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.973513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05180a-6892-4202-8f01-138449bdce07 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.974237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05180a-6892-4202-8f01-138449bdce07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:28.974538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05180a-6892-4202-8f01-138449bdce07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:28.975923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05180a-6892-4202-8f01-138449bdce07 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:28.984281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 110/220] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:28.985542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe621f-5447-46fe-beef-f49b9f5d9f97 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:28.985829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe621f-5447-46fe-beef-f49b9f5d9f97 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:28.986101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe621f-5447-46fe-beef-f49b9f5d9f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:28.991072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:29.002793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cdbe621f-5447-46fe-beef-f49b9f5d9f97 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:29.003960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cdbe621f-5447-46fe-beef-f49b9f5d9f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:29.013401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02657e1-40e1-4838-b540-7305e47ca5b1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.036152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:29.046367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.047053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.047341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.047620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.048406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.059180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.070764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 111/221] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:29.072182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe621f-5447-46fe-beef-f49b9f5d9f97 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.168109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe621f-5447-46fe-beef-f49b9f5d9f97 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.202679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:29.207912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.208828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.209111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.209734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] PATH_INFO: `/v3/users/979c748c6e1b43b3bdb79455171dcb10` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.210608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] RBAC: Authorizing `identity:get_user(user_id=979c748c6e1b43b3bdb79455171dcb10)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.212124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.213799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 112/222] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/users/979c748c6e1b43b3bdb79455171dcb10 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:29.214553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.230131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 111/223] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:29:28 2022] POST /identity/v3/users => generated 297 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:29.241790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.328657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923beaa1-b545-4dd7-9c84-f617597fd5ba None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.351806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d0c5-046e-4ea5-90d0-eb5488da59f0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.367559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.368617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.368980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.369253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] PATH_INFO: `/v3/users/47e860cee33d4c12884dc801a6c2c83a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.385995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] RBAC: Authorizing `identity:get_user(user_id=47e860cee33d4c12884dc801a6c2c83a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.388081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.388809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.389252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 113/224] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/users/47e860cee33d4c12884dc801a6c2c83a => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:29.395062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.395062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.395255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.395327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.403886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:29.412301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:29.413267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 112/225] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:29.414365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.463043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:29.467390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:29.522781 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b808-afd6-4a1f-b522-1d07925375cd None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.546175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd9bfa-ec80-4c9e-a575-0ca9f48b44fc None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.566348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.567318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.567597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.567871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.569158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.570649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.576475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 114/226] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:29.576475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.601471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.602913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.603445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.603860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.684954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c821c-1c58-4618-9f44-2d79b3ff01b6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.693585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.695719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.719336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:29.732121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.733193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.735739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.735739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.735739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:29.735739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.736324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.745331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:29.756532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 115/227] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:29.757991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.815077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 113/228] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:29.815856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.880031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e17c6-c645-4f0b-921d-97fc7bd6cede None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.917568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.918507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.918790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.919064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/979c748c6e1b43b3bdb79455171dcb10/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.921208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f8537768c1b489d865a39e704eb9b79, user_id=979c748c6e1b43b3bdb79455171dcb10, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.922588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.940936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eccc4-d587-420c-8bfb-02e3ed3b41e6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:29.974919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:29.975747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:29.976053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:29.976300 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] PATH_INFO: `/v3/users/979c748c6e1b43b3bdb79455171dcb10` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:29.978079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] RBAC: Authorizing `identity:get_user(user_id=979c748c6e1b43b3bdb79455171dcb10)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:29.979462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:29.984792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 114/229] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/users/979c748c6e1b43b3bdb79455171dcb10 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:29.984792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.993984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 116/230] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:29 2022] PUT /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/979c748c6e1b43b3bdb79455171dcb10/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:29.995047 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:29.996734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:30.051060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:30.106642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65f2b-d2eb-46c2-9591-d69a8b61138b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:30.109610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d667d-2f8a-4bec-bd72-fc8fde431ce6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:30.146750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:30.146962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:30.147755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:30.148039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:30.148314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:30.148783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:30.149256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:30.149685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:30.149970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:30.151154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:30.151679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:30.153153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 115/231] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:30.153840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:30.155667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 117/232] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:29 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:30.161627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:30.165750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:30.250903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:30.277437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7704952-4cb4-491d-9417-2243c6e67692 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:30.303220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785bbb-2ad9-4d0b-82a0-b88b9638eb67 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:30.318365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc229ba-284b-4a1b-942e-1020b902ed67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:30.319315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc229ba-284b-4a1b-942e-1020b902ed67 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:30.320414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:30.320910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc229ba-284b-4a1b-942e-1020b902ed67 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:30.322738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc229ba-284b-4a1b-942e-1020b902ed67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:30.323780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc229ba-284b-4a1b-942e-1020b902ed67 None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=47e860cee33d4c12884dc801a6c2c83a) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:30.324328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc229ba-284b-4a1b-942e-1020b902ed67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:30.326097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc229ba-284b-4a1b-942e-1020b902ed67 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:30.333345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 116/233] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:30 2022] GET /identity/v3/role_assignments?user.id=47e860cee33d4c12884dc801a6c2c83a&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:30.341378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d1dda-e139-4ae8-b387-08e45b77d190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:30.342318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d1dda-e139-4ae8-b387-08e45b77d190 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:30.342578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d1dda-e139-4ae8-b387-08e45b77d190 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:30.342849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d1dda-e139-4ae8-b387-08e45b77d190 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/979c748c6e1b43b3bdb79455171dcb10/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:30.345245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d1dda-e139-4ae8-b387-08e45b77d190 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f8537768c1b489d865a39e704eb9b79, user_id=979c748c6e1b43b3bdb79455171dcb10, role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:30.346815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d1dda-e139-4ae8-b387-08e45b77d190 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:30.443340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 118/234] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:30 2022] PUT /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/979c748c6e1b43b3bdb79455171dcb10/roles/4182f79f5cda459497c049f3f07a1713 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:31.617762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c9ea9-1406-44d8-b64e-4ec5b5b1188a None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:31.618300 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c9ea9-1406-44d8-b64e-4ec5b5b1188a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:31.618403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c9ea9-1406-44d8-b64e-4ec5b5b1188a None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:31.620004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 117/235] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:31.625484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad37904-c01a-4900-90f8-5ad89bc004d5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:31.625886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad37904-c01a-4900-90f8-5ad89bc004d5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:31.626052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad37904-c01a-4900-90f8-5ad89bc004d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:31.649231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bad37904-c01a-4900-90f8-5ad89bc004d5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:31.651011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bad37904-c01a-4900-90f8-5ad89bc004d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:31.738726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 119/236] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:31.777922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c9ea9-1406-44d8-b64e-4ec5b5b1188a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:31.923790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c9ea9-1406-44d8-b64e-4ec5b5b1188a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:31.968795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:31.969598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:31.969887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:31.970165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:31.984291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:31.988187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:31.996506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:31.997384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:31.998514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:32.002713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 118/237] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:32.009602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad37904-c01a-4900-90f8-5ad89bc004d5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:32.141974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad37904-c01a-4900-90f8-5ad89bc004d5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:32.149679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:32.184585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799914b-5cc0-480f-9613-4d061148c63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:32.185190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799914b-5cc0-480f-9613-4d061148c63e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.185481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799914b-5cc0-480f-9613-4d061148c63e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.185771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799914b-5cc0-480f-9613-4d061148c63e None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.186923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799914b-5cc0-480f-9613-4d061148c63e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:32.192879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799914b-5cc0-480f-9613-4d061148c63e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:32.194308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799914b-5cc0-480f-9613-4d061148c63e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:32.203877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 120/238] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:32.209407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:32.275111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48968cd-c143-4e85-9a99-aa6bf154cf09 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.275431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48968cd-c143-4e85-9a99-aa6bf154cf09 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.275671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48968cd-c143-4e85-9a99-aa6bf154cf09 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.277347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 121/239] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:32.284240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f19fb-635a-4ad5-acf4-8ee3784d7a2b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.284240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f19fb-635a-4ad5-acf4-8ee3784d7a2b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.284240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f19fb-635a-4ad5-acf4-8ee3784d7a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.303056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-2c4f19fb-635a-4ad5-acf4-8ee3784d7a2b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:32.304673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2c4f19fb-635a-4ad5-acf4-8ee3784d7a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:32.347608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c594-a74a-4f31-9c1f-1939db52b36e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:32.386496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:32.388682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.389328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.390823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] PATH_INFO: `/v3/users/2895d98f8f8846379d71a0f4f0740b5f` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.417886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] RBAC: Authorizing `identity:get_user(user_id=2895d98f8f8846379d71a0f4f0740b5f)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:32.424243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:32.426203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 119/240] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:32 2022] GET /identity/v3/users/2895d98f8f8846379d71a0f4f0740b5f => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:32.434698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:32.467499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 122/241] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:32.492404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f19fb-635a-4ad5-acf4-8ee3784d7a2b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:32.547692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:32.598333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4804d-1ede-445a-9b62-428b54db8d6e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:32.617332 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:32.622147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f19fb-635a-4ad5-acf4-8ee3784d7a2b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:32.635400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc4188-fb21-43d8-b568-1ebc6505537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:32.636055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4188-fb21-43d8-b568-1ebc6505537e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.636371 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4188-fb21-43d8-b568-1ebc6505537e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.636678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc4188-fb21-43d8-b568-1ebc6505537e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.659747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c3251-3227-470b-803d-f56f4283c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:32.660670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3251-3227-470b-803d-f56f4283c7cd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.660957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3251-3227-470b-803d-f56f4283c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.661235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3251-3227-470b-803d-f56f4283c7cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.662408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c3251-3227-470b-803d-f56f4283c7cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:32.663871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c3251-3227-470b-803d-f56f4283c7cd None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:32.666269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 123/242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:32.667463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d3a82-ac13-48a7-af2a-e3edd6e86c5c None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.667749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d3a82-ac13-48a7-af2a-e3edd6e86c5c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.668029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d3a82-ac13-48a7-af2a-e3edd6e86c5c None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.669427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 124/243] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:32.670577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2025c0-4fad-49d2-ad8f-cc966d3aa6ac None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.670870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2025c0-4fad-49d2-ad8f-cc966d3aa6ac None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.671148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2025c0-4fad-49d2-ad8f-cc966d3aa6ac None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.672661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 125/244] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:32.673644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2025c0-4fad-49d2-ad8f-cc966d3aa6ac None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:32.712427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc4188-fb21-43d8-b568-1ebc6505537e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:32.713840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc4188-fb21-43d8-b568-1ebc6505537e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:32.771430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:32.783887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2025c0-4fad-49d2-ad8f-cc966d3aa6ac None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:32.815848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 120/245] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:32.817264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4ab4-b53f-4835-8fde-c0bb925c3600 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.817630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4ab4-b53f-4835-8fde-c0bb925c3600 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.817988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4ab4-b53f-4835-8fde-c0bb925c3600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.823579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e321e-a93e-40dd-87c1-87c882937af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:32.824257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e321e-a93e-40dd-87c1-87c882937af1 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.824576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e321e-a93e-40dd-87c1-87c882937af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.824861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e321e-a93e-40dd-87c1-87c882937af1 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.825581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e321e-a93e-40dd-87c1-87c882937af1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:32.827135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e321e-a93e-40dd-87c1-87c882937af1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:32.836178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-478a4ab4-b53f-4835-8fde-c0bb925c3600 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:32.837963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-478a4ab4-b53f-4835-8fde-c0bb925c3600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:32.895572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 126/246] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:29:32 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:32.897176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0484a3-3cdb-40e6-8929-ed692b689b79 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:32.897547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0484a3-3cdb-40e6-8929-ed692b689b79 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:32.898076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0484a3-3cdb-40e6-8929-ed692b689b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:32.924464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ac0484a3-3cdb-40e6-8929-ed692b689b79 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:32.924464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ac0484a3-3cdb-40e6-8929-ed692b689b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:32.925349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 121/247] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:32.926513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a4ab4-b53f-4835-8fde-c0bb925c3600 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:32.947338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.016558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 127/248] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:33.017773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0484a3-3cdb-40e6-8929-ed692b689b79 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.070551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.083870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a4ab4-b53f-4835-8fde-c0bb925c3600 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.129737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.130960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.131446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.131919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.133621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:33.135829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:33.137622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 122/249] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:32 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:33.138388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.170231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0484a3-3cdb-40e6-8929-ed692b689b79 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.219343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffefcb-37f1-4a64-bb8d-1ac436f0dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.220244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffefcb-37f1-4a64-bb8d-1ac436f0dc64 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.220557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffefcb-37f1-4a64-bb8d-1ac436f0dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.220838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffefcb-37f1-4a64-bb8d-1ac436f0dc64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.234134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffefcb-37f1-4a64-bb8d-1ac436f0dc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-94ffefcb-37f1-4a64-bb8d-1ac436f0dc64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:33.241966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:33.242878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 128/250] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:33.244811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4cdc-a41d-4b15-add1-dbc9421fb43d None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.245514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4cdc-a41d-4b15-add1-dbc9421fb43d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.245514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d4cdc-a41d-4b15-add1-dbc9421fb43d None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.246794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 129/251] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:33.247323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4cdc-a41d-4b15-add1-dbc9421fb43d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.263177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41270f9f-d493-4128-b1be-1927c0fd228d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.267649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.302752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.303654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.304009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.304355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.318091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:33.326357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:33.327277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:33.328115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 123/252] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:33.328804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.349875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.381932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d4cdc-a41d-4b15-add1-dbc9421fb43d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.422101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de366bdb-26b1-456a-ac84-8b395aadf3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.423108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de366bdb-26b1-456a-ac84-8b395aadf3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.423589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de366bdb-26b1-456a-ac84-8b395aadf3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.424078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de366bdb-26b1-456a-ac84-8b395aadf3a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.425177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de366bdb-26b1-456a-ac84-8b395aadf3a1 None admin] RBAC: Adding query filter params (role.id=a3a50df9b07a42c4b1361e06ea200403, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=2895d98f8f8846379d71a0f4f0740b5f) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:33.425752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de366bdb-26b1-456a-ac84-8b395aadf3a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:33.427474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de366bdb-26b1-456a-ac84-8b395aadf3a1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:33.434633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 130/253] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/role_assignments?user.id=2895d98f8f8846379d71a0f4f0740b5f&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=a3a50df9b07a42c4b1361e06ea200403 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:33.436104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f04df6-3335-4b31-b936-2b9b479b9f93 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.436599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f04df6-3335-4b31-b936-2b9b479b9f93 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.437105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f04df6-3335-4b31-b936-2b9b479b9f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.439075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96c031-ab88-4f09-900e-591ea7c30842 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.455041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d8f04df6-3335-4b31-b936-2b9b479b9f93 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:33.456566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d8f04df6-3335-4b31-b936-2b9b479b9f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:33.490254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.491097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.491471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.491850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.492960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:33.493203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:33.494700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:33.512719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 124/254] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:33.513515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.535903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 131/255] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:33.537088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f04df6-3335-4b31-b936-2b9b479b9f93 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.546616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.650004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.673525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93623e-fd65-4cb3-948b-9c604abbd435 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.693080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f04df6-3335-4b31-b936-2b9b479b9f93 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.711171 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.711995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.712367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.712744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.713558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:33.713934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:33.715380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:33.724234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 125/256] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:33.725140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.746110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.748602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.749284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.750392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] PATH_INFO: `/v3/users/979c748c6e1b43b3bdb79455171dcb10` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.752776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] RBAC: Authorizing `identity:get_user(user_id=979c748c6e1b43b3bdb79455171dcb10)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:33.754586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:33.757683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 132/257] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/users/979c748c6e1b43b3bdb79455171dcb10 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:33.760029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.879563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.902839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76cc86-cf75-4585-98d9-2571c04af8da None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.908691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b93fff-02df-4543-844b-e09accb4290a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:33.969660 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.970858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.971142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.971418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.975803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:33.976693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:33.978321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:33.978321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] PATH_INFO: `/v3/users/47e860cee33d4c12884dc801a6c2c83a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:33.979713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] RBAC: Authorizing `identity:get_user(user_id=47e860cee33d4c12884dc801a6c2c83a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:33.981180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:33.985560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 133/258] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/users/47e860cee33d4c12884dc801a6c2c83a => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:33.986337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:33.989792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:33.995043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:34.011063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:34.011994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:34.012828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:34.014590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 126/259] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:34.015393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:34.170657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a04c35-8920-4373-bcbb-0a7ecf970667 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:34.198331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15610f7-d623-471b-8701-d75f3e40ea51 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:34.207584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:34.208346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:34.208749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:34.209117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:34.223123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:34.289339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:34.290275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:34.290560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:34.290841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:34.292944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:34.294423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:34.296157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 127/260] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:34 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:34.296952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:34.299408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:34.300904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:34.404813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:34.433947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9687465-dbdc-494b-8598-89c3decdb912 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:34.486988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:34.487669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:34.487957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:34.488240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:34.489027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:34.489338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:34.490737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:34.513987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 134/262] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:34.514733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:34.515187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 128/262] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:34.516116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:34.592849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:34.697298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d697-a975-4c7d-ac1a-7bc13c74c9b1 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:34.707942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c653f0c-522d-49e9-909b-20ff0f0d9e06 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:34.763082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:34.763649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:34.763989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:34.765111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:34.765404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/47e860cee33d4c12884dc801a6c2c83a/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:34.768104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] RBAC: Authorizing `identity:create_grant(project_id=4f8537768c1b489d865a39e704eb9b79, user_id=47e860cee33d4c12884dc801a6c2c83a, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:34.769603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:34.774382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:34.776734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:34.776734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:34.776734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] PATH_INFO: `/v3/users/979c748c6e1b43b3bdb79455171dcb10` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:34.778302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] RBAC: Authorizing `identity:get_user(user_id=979c748c6e1b43b3bdb79455171dcb10)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:34.780286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:34.782569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 129/263] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:34 2022] GET /identity/v3/users/979c748c6e1b43b3bdb79455171dcb10 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:34.783049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:34.859041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 135/264] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:34 2022] PUT /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/47e860cee33d4c12884dc801a6c2c83a/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:34.860230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:35.019627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0ccac-2a61-4ebc-a6d6-1bb1dd9b1e21 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:35.099263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:35.100387 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.101034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.101034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.108422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:35.109945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:35.111992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 130/265] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:34 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:35.118907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:35.175090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f321ee-fc6e-4b62-ae91-d35f2946bf2d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:35.192823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:35.269112 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65052f0c-6095-4c36-b717-8aa729c2dfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:35.269829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052f0c-6095-4c36-b717-8aa729c2dfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.270060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052f0c-6095-4c36-b717-8aa729c2dfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.270349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052f0c-6095-4c36-b717-8aa729c2dfa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.277808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:35.330418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5270782-f43d-49e3-a026-1f4e033dd3ca None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:35.372870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:35.373143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65052f0c-6095-4c36-b717-8aa729c2dfa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:35.373635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.373931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.374216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.374487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65052f0c-6095-4c36-b717-8aa729c2dfa4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:35.375066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=979c748c6e1b43b3bdb79455171dcb10) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:35.375384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:35.377124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:35.384578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 131/266] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:35 2022] GET /identity/v3/role_assignments?user.id=979c748c6e1b43b3bdb79455171dcb10&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:35.499059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 136/267] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:35.504740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:35.604291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7938bb-7e52-455e-940f-0a808d6d6bfc None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.604625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7938bb-7e52-455e-940f-0a808d6d6bfc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.604827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7938bb-7e52-455e-940f-0a808d6d6bfc None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.606486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 137/268] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:35.611745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aaa8e-57e7-45be-95d4-bec43650983f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.612032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aaa8e-57e7-45be-95d4-bec43650983f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.612316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aaa8e-57e7-45be-95d4-bec43650983f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.634829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:35.661531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8e0aaa8e-57e7-45be-95d4-bec43650983f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:35.662830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8e0aaa8e-57e7-45be-95d4-bec43650983f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:35.686687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c869f18-c6e7-4389-acd8-b215ff3ccd59 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:35.749359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:35.760076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658ea41-5e70-46f6-ac96-1e353dbd01e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:35.761331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658ea41-5e70-46f6-ac96-1e353dbd01e8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.761771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658ea41-5e70-46f6-ac96-1e353dbd01e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.773411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658ea41-5e70-46f6-ac96-1e353dbd01e8 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.773411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7658ea41-5e70-46f6-ac96-1e353dbd01e8 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:35.773411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7658ea41-5e70-46f6-ac96-1e353dbd01e8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:35.773411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 132/269] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:35 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:35.773411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa5abb-1f9d-4721-b059-c34b6ff7fbf1 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.773411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa5abb-1f9d-4721-b059-c34b6ff7fbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.773859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa5abb-1f9d-4721-b059-c34b6ff7fbf1 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.773859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 133/270] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:35.775018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa5abb-1f9d-4721-b059-c34b6ff7fbf1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:35.798980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 138/271] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 17:29:35.801008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fcaaa-a307-407c-a6a9-36f9ba44d633 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.801469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fcaaa-a307-407c-a6a9-36f9ba44d633 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.801882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fcaaa-a307-407c-a6a9-36f9ba44d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.822102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a11fcaaa-a307-407c-a6a9-36f9ba44d633 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:35.823604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a11fcaaa-a307-407c-a6a9-36f9ba44d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:35.912539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 139/272] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:35.913711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fcaaa-a307-407c-a6a9-36f9ba44d633 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:35.925452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa5abb-1f9d-4721-b059-c34b6ff7fbf1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:35.972896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:35.973875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:35.974306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:35.974749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:35.975740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] RBAC: Adding query filter params (role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=979c748c6e1b43b3bdb79455171dcb10) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:35.976667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:35.979200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:35.987739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 134/273] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:35 2022] GET /identity/v3/role_assignments?user.id=979c748c6e1b43b3bdb79455171dcb10&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=4182f79f5cda459497c049f3f07a1713 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:35.989180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.034387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:36.079411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:36.098513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fcaaa-a307-407c-a6a9-36f9ba44d633 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.121744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e739f6-87e1-4ed8-bd07-24b12e061d92 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.139209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444a1d2-1e11-4976-b5b9-00348a107559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.140686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a1d2-1e11-4976-b5b9-00348a107559 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.141218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a1d2-1e11-4976-b5b9-00348a107559 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.141720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444a1d2-1e11-4976-b5b9-00348a107559 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.143160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444a1d2-1e11-4976-b5b9-00348a107559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:36.145036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444a1d2-1e11-4976-b5b9-00348a107559 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:36.147481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 140/274] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:36.153861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a22a-b5e9-466d-bb1b-4d55c4aa7940 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.154427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a22a-b5e9-466d-bb1b-4d55c4aa7940 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.154900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158a22a-b5e9-466d-bb1b-4d55c4aa7940 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.157678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 141/275] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:36.158664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158a22a-b5e9-466d-bb1b-4d55c4aa7940 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.163953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1a146-d898-4eb2-aa3c-33aff843c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.163953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1a146-d898-4eb2-aa3c-33aff843c9ca None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.164173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1a146-d898-4eb2-aa3c-33aff843c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.164464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1a146-d898-4eb2-aa3c-33aff843c9ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.178568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1a146-d898-4eb2-aa3c-33aff843c9ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-7ef1a146-d898-4eb2-aa3c-33aff843c9ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:36.186976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:36.188055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:36.189110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 135/276] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:36.189920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6045b-1270-47a5-9ac3-0428a0b2a7d7 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.189920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6045b-1270-47a5-9ac3-0428a0b2a7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.190038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6045b-1270-47a5-9ac3-0428a0b2a7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.207915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-52d6045b-1270-47a5-9ac3-0428a0b2a7d7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:36.209129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-52d6045b-1270-47a5-9ac3-0428a0b2a7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:36.285206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158a22a-b5e9-466d-bb1b-4d55c4aa7940 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.285909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 136/277] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:36.286769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6045b-1270-47a5-9ac3-0428a0b2a7d7 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.324755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:36.331319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.333724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.333724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.333724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.333724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:36.334998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:36.342463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:36.387064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6045b-1270-47a5-9ac3-0428a0b2a7d7 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.420025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.420670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.420959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.421231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.421980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:36.422386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:36.423660 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:36.436833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 142/278] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:29:36 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:36.436833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.441520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 137/279] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:36.447376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.572948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10e5a-1a81-4eee-87a6-8f9c4b00bc63 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.580035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c4ebc-36d6-4e93-aff6-842dc0132855 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.593918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:36.621804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.622951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.623237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.623512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] PATH_INFO: `/v3/users/817126fb17764b4e98996d449e861810` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.626246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.627031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.627322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.627600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.640849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] RBAC: Authorizing `identity:get_user(user_id=817126fb17764b4e98996d449e861810)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:36.641026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:36.642309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:36.644194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 138/280] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:36 2022] GET /identity/v3/users/817126fb17764b4e98996d449e861810 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:36.649589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:36.650130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:36.651013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:36.651805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 143/281] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:36.657798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.723189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:36.782327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601e5a1-759e-4f86-8de9-ce6f8e541a6d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.793968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d3e27-443c-4979-8c56-94514a10a13b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:36.843081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.843843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.844133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.844420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.845238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:36.845519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:36.845849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:36.846587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:36.846913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:36.847010 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:36.847251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:36.848198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, user.id=817126fb17764b4e98996d449e861810, scope.system=all) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:36.848638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:36.850240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:36.857550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 139/282] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jul 20 17:29:36 2022] GET /identity/v3/role_assignments?user.id=817126fb17764b4e98996d449e861810&scope.system=all&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:36.858270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 144/283] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:36.864753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:36.910721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:36.991502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c2841-9778-4b34-b4a0-c6412082478c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:37.034134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:37.035018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:37.035305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:37.035580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] PATH_INFO: `/v3/users/2895d98f8f8846379d71a0f4f0740b5f` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:37.036832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] RBAC: Authorizing `identity:get_user(user_id=2895d98f8f8846379d71a0f4f0740b5f)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:37.038298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:37.039980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 140/284] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:36 2022] GET /identity/v3/users/2895d98f8f8846379d71a0f4f0740b5f => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:37.045686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:37.206240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02bd7-ae60-4d0b-9328-ff6b2a6079c3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:37.216648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:37.248382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4249a5c-7f4b-4655-8318-bb89450e70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:37.249217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4249a5c-7f4b-4655-8318-bb89450e70ad None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:37.249502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4249a5c-7f4b-4655-8318-bb89450e70ad None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:37.249771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4249a5c-7f4b-4655-8318-bb89450e70ad None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:37.251072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4249a5c-7f4b-4655-8318-bb89450e70ad None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:37.252784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4249a5c-7f4b-4655-8318-bb89450e70ad None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:37.254806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 145/285] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:37 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:37.261540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:37.383137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b989cd3-c3d2-4dbc-9908-d089f7e70c0e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:37.386043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:37.418884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d19ca3-e821-4370-a664-77b2fe221239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:37.419765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19ca3-e821-4370-a664-77b2fe221239 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:37.420045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19ca3-e821-4370-a664-77b2fe221239 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:37.420316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19ca3-e821-4370-a664-77b2fe221239 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/2895d98f8f8846379d71a0f4f0740b5f/roles/a3a50df9b07a42c4b1361e06ea200403` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:37.422526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d19ca3-e821-4370-a664-77b2fe221239 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f8537768c1b489d865a39e704eb9b79, user_id=2895d98f8f8846379d71a0f4f0740b5f, role_id=a3a50df9b07a42c4b1361e06ea200403)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:37.424312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d19ca3-e821-4370-a664-77b2fe221239 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:37.533322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 141/286] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:29:37 2022] PUT /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79/users/2895d98f8f8846379d71a0f4f0740b5f/roles/a3a50df9b07a42c4b1361e06ea200403 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:37.654234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43329-6ece-45f9-b3fe-9eed45b4967a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:37.654614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43329-6ece-45f9-b3fe-9eed45b4967a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:37.654798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43329-6ece-45f9-b3fe-9eed45b4967a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:37.656401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 146/287] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:37.662413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b57d71-a510-489a-b72a-b5285082fdac None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:37.662784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b57d71-a510-489a-b72a-b5285082fdac None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:37.663134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b57d71-a510-489a-b72a-b5285082fdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:37.693909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-71b57d71-a510-489a-b72a-b5285082fdac None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:37.695526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-71b57d71-a510-489a-b72a-b5285082fdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:37.788553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 142/288] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:37.813462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43329-6ece-45f9-b3fe-9eed45b4967a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:37.929917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43329-6ece-45f9-b3fe-9eed45b4967a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:37.977057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:37.978323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:37.978877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:37.979393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:37.994094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:38.002654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:38.003529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:38.004319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:38.006760 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 147/289] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:38.013394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b57d71-a510-489a-b72a-b5285082fdac None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:38.154848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:38.161612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b57d71-a510-489a-b72a-b5285082fdac None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:38.214795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:38.215501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:38.215790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:38.216071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:38.216899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:38.217684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:38.219127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:38.233148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 143/290] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:38.238329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:38.291488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:38.360947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7f4d6-2f16-4cf9-bf5b-55abd5256e69 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:38.397416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:38.398659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:38.399161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:38.399665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] PATH_INFO: `/v3/users/47e860cee33d4c12884dc801a6c2c83a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:38.401283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] RBAC: Authorizing `identity:get_user(user_id=47e860cee33d4c12884dc801a6c2c83a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:38.402995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:38.405152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 148/291] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:38 2022] GET /identity/v3/users/47e860cee33d4c12884dc801a6c2c83a => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:38.410617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:38.558532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2ad91-5de5-4f86-87e5-c120e2c41959 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:38.596852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4213a4-c8a2-40bf-8a56-f7eb3f888779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:38.597502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4213a4-c8a2-40bf-8a56-f7eb3f888779 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:38.597781 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4213a4-c8a2-40bf-8a56-f7eb3f888779 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:38.598055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4213a4-c8a2-40bf-8a56-f7eb3f888779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:38.630896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:38.686461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4213a4-c8a2-40bf-8a56-f7eb3f888779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:38.687917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4213a4-c8a2-40bf-8a56-f7eb3f888779 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:38.800944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 144/292] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:38.807068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:38.954883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f48b0-ffad-4529-a301-0bb5bcf78857 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:38.962220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:38.999938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34c5b-266b-42ea-abe2-99343531fb43 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.000237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34c5b-266b-42ea-abe2-99343531fb43 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.000507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34c5b-266b-42ea-abe2-99343531fb43 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.002121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 145/293] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:39.005838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:39.006706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.010886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.010886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.010886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:39.010886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:39.012760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5dbfd-2917-495c-97ab-077e0dcb0ebc None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.013041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5dbfd-2917-495c-97ab-077e0dcb0ebc None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.013316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5dbfd-2917-495c-97ab-077e0dcb0ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.019819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 149/294] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:38 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:39.029637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.045793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-87f5dbfd-2917-495c-97ab-077e0dcb0ebc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:39.046988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-87f5dbfd-2917-495c-97ab-077e0dcb0ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:39.113314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:39.131705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 146/295] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:39.157934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5dbfd-2917-495c-97ab-077e0dcb0ebc None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.259276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11bd9-9dd2-4037-be6f-0d56c9c1f4df None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:39.263081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5dbfd-2917-495c-97ab-077e0dcb0ebc None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:39.299415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4f85f-c4fe-4397-8a6d-75b096b92d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:39.300233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4f85f-c4fe-4397-8a6d-75b096b92d40 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.300556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4f85f-c4fe-4397-8a6d-75b096b92d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.300838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4f85f-c4fe-4397-8a6d-75b096b92d40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.311957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4f85f-c4fe-4397-8a6d-75b096b92d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:39.316699 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569ad99-bea2-4434-af56-c6020d14b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:39.317578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569ad99-bea2-4434-af56-c6020d14b18d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.318059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569ad99-bea2-4434-af56-c6020d14b18d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.318525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569ad99-bea2-4434-af56-c6020d14b18d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.319586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569ad99-bea2-4434-af56-c6020d14b18d None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=47e860cee33d4c12884dc801a6c2c83a) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:39.320086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569ad99-bea2-4434-af56-c6020d14b18d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-66f4f85f-c4fe-4397-8a6d-75b096b92d40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:39.320197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:39.321235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:39.322013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 147/296] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:39.323068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ae2a-9c70-43c9-bd3a-acb705005bb8 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.324748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569ad99-bea2-4434-af56-c6020d14b18d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:39.327526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ae2a-9c70-43c9-bd3a-acb705005bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.327865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ae2a-9c70-43c9-bd3a-acb705005bb8 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.329268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 148/297] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:39.333547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4ae2a-9c70-43c9-bd3a-acb705005bb8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.337851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 150/298] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/role_assignments?user.id=47e860cee33d4c12884dc801a6c2c83a&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:39.339161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaee0c7-3c99-40ce-a863-45d0a0475107 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.339449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaee0c7-3c99-40ce-a863-45d0a0475107 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.339726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaee0c7-3c99-40ce-a863-45d0a0475107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.357840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-afaee0c7-3c99-40ce-a863-45d0a0475107 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:39.359169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-afaee0c7-3c99-40ce-a863-45d0a0475107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:39.421218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:39.431307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:39.499213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 151/299] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:39.525557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4ae2a-9c70-43c9-bd3a-acb705005bb8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:39.539973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855811f-666a-4c51-b435-2b289b72cbdb None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.540285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855811f-666a-4c51-b435-2b289b72cbdb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.540571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855811f-666a-4c51-b435-2b289b72cbdb None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.543176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 152/300] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:39.549809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855811f-666a-4c51-b435-2b289b72cbdb None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.576054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00eaf3-16ac-4097-9ca3-c280f84c32ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:39.577033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00eaf3-16ac-4097-9ca3-c280f84c32ae None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.577323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00eaf3-16ac-4097-9ca3-c280f84c32ae None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.577603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00eaf3-16ac-4097-9ca3-c280f84c32ae None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.578602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00eaf3-16ac-4097-9ca3-c280f84c32ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:39.579315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00eaf3-16ac-4097-9ca3-c280f84c32ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:39.581018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00eaf3-16ac-4097-9ca3-c280f84c32ae None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:39.591795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 149/301] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:39.593370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0de29-2a48-4cc3-99aa-7feabe5ac695 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.593663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0de29-2a48-4cc3-99aa-7feabe5ac695 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.593967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0de29-2a48-4cc3-99aa-7feabe5ac695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.613363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1ab0de29-2a48-4cc3-99aa-7feabe5ac695 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:39.614556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1ab0de29-2a48-4cc3-99aa-7feabe5ac695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:39.675192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855811f-666a-4c51-b435-2b289b72cbdb None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:39.687209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 150/302] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:39.697179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0de29-2a48-4cc3-99aa-7feabe5ac695 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.713307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:39.714107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.714390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.714681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.724905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:39.733057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:39.733938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 153/303] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:39.739879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.745996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:39.766818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:39.807625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0de29-2a48-4cc3-99aa-7feabe5ac695 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:39.843742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7839a-9b73-4187-96af-be9eedc64d7c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:39.844745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:39.845778 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.846819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.847103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] PATH_INFO: `/v3/users/1d993b1406894111889972ce58482a68` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.862103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] RBAC: Authorizing `identity:get_user(user_id=1d993b1406894111889972ce58482a68)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:39.863852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:39.871080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 151/304] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/users/1d993b1406894111889972ce58482a68 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:39.871080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.880209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:39.881031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:39.881315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:39.881588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:39.883046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:39.884573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:39.886609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 154/305] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:39.887433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:39.979326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:39.983972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:39.985995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e5d13-e386-40e4-8ebc-cd0827447e7e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:39.990244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068193c-951d-40e8-bf68-3184224c5bf9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:40.022028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.022778 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.023077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.023192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.023379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.023805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.024110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.024227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:40.024464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.024558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:40.027626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] RBAC: Adding query filter params (role.id=a3a50df9b07a42c4b1361e06ea200403, user.id=1d993b1406894111889972ce58482a68, scope.system=all) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:40.027626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:40.027797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:40.027879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:40.032530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 155/306] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/role_assignments?user.id=1d993b1406894111889972ce58482a68&scope.system=all&role.id=a3a50df9b07a42c4b1361e06ea200403 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:40.033365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:40.044321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 152/307] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:40.045663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:40.159250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eaaee-e012-48fa-b780-e16f3ea5ad70 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:40.165611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6710122-2b54-481c-9c56-e9a3746dbd87 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:40.199049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd8484-33ba-4577-a42c-2eec36783dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.200679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8484-33ba-4577-a42c-2eec36783dae None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.200679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8484-33ba-4577-a42c-2eec36783dae None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.200679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8484-33ba-4577-a42c-2eec36783dae None admin] PATH_INFO: `/v3/groups` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.202078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd8484-33ba-4577-a42c-2eec36783dae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:40.203859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd8484-33ba-4577-a42c-2eec36783dae None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:40.213350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.214248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.214537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.214813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] PATH_INFO: `/v3/users/817126fb17764b4e98996d449e861810` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.216034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] RBAC: Authorizing `identity:get_user(user_id=817126fb17764b4e98996d449e861810)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:40.217570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:40.219419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 153/308] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:40 2022] GET /identity/v3/users/817126fb17764b4e98996d449e861810 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:40.225013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:40.254386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:40.276573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 156/309] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Jul 20 17:29:40 2022] POST /identity/v3/groups => generated 229 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:40.313448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41cfb3-96b4-43c0-b0e2-ff9fa2835f4f None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.313764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41cfb3-96b4-43c0-b0e2-ff9fa2835f4f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.314019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f41cfb3-96b4-43c0-b0e2-ff9fa2835f4f None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.315726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 157/310] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:40.321351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9452a9-02b1-4a6f-90da-504da6fb941b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.321670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9452a9-02b1-4a6f-90da-504da6fb941b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.321927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9452a9-02b1-4a6f-90da-504da6fb941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.340233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-9f9452a9-02b1-4a6f-90da-504da6fb941b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:40.341555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9f9452a9-02b1-4a6f-90da-504da6fb941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:40.349095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e8f92-6468-4fac-b7fb-5601b2b83cbf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:40.357835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:40.391712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.392647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.392940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.393226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] PATH_INFO: `/v3/system/users/817126fb17764b4e98996d449e861810/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.395115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=817126fb17764b4e98996d449e861810, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:40.396586 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:40.420999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 158/311] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:40.444621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9452a9-02b1-4a6f-90da-504da6fb941b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:40.464047 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 154/312] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Jul 20 17:29:40 2022] PUT /identity/v3/system/users/817126fb17764b4e98996d449e861810/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:29:40.565500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9452a9-02b1-4a6f-90da-504da6fb941b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:40.607416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.608154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.608451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.608748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.621045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:40.628917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:40.629821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:40.630633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 159/313] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:40.636419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:40.734515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:40.744261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00720870-daa2-455e-8ba2-83605100aa4a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:40.782676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341703-9969-4b23-9169-f98c09043714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.783421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341703-9969-4b23-9169-f98c09043714 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.783773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341703-9969-4b23-9169-f98c09043714 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.784057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18341703-9969-4b23-9169-f98c09043714 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.784859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341703-9969-4b23-9169-f98c09043714 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:40.785169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341703-9969-4b23-9169-f98c09043714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:40.786558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18341703-9969-4b23-9169-f98c09043714 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:40.796601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 155/314] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:40.801722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:40.818311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:40.914645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96e03b-d585-476d-9f6b-eb7ec6b56595 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:40.948507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:40.949385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:40.949764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:40.950051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] PATH_INFO: `/v3/users/2895d98f8f8846379d71a0f4f0740b5f` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:40.951259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=2895d98f8f8846379d71a0f4f0740b5f)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:40.952744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:40.954677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 160/315] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:40 2022] GET /identity/v3/users/2895d98f8f8846379d71a0f4f0740b5f => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:40.960142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341703-9969-4b23-9169-f98c09043714 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:41.070232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18341703-9969-4b23-9169-f98c09043714 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:41.109209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:41.110100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:41.110528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:41.110979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:41.133660 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:41.182277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:41.184026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:41.302220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 156/316] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:41.307512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:41.391001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:41.415448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4516f19-e9e4-4086-9008-217c28801fa8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:41.449200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcf1d8-486a-414a-9beb-a44c8726ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:41.450259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcf1d8-486a-414a-9beb-a44c8726ebde None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:41.450686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcf1d8-486a-414a-9beb-a44c8726ebde None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:41.451200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcf1d8-486a-414a-9beb-a44c8726ebde None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:41.452557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcf1d8-486a-414a-9beb-a44c8726ebde None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:41.454135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcf1d8-486a-414a-9beb-a44c8726ebde None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:41.456319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 161/317] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:41 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:41.465502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:41.520314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:41.570272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fad648-4aa5-41c3-8ac1-93a5d717684a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:41.606786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e74f07-1770-4e67-84a2-7d1ab1b73c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:41.607733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e74f07-1770-4e67-84a2-7d1ab1b73c3f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:41.608159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e74f07-1770-4e67-84a2-7d1ab1b73c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:41.608641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e74f07-1770-4e67-84a2-7d1ab1b73c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:41.609621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e74f07-1770-4e67-84a2-7d1ab1b73c3f None admin] RBAC: Adding query filter params (role.id=a3a50df9b07a42c4b1361e06ea200403, scope.project.id=4f8537768c1b489d865a39e704eb9b79, user.id=2895d98f8f8846379d71a0f4f0740b5f) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:41.610075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e74f07-1770-4e67-84a2-7d1ab1b73c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:41.611682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e74f07-1770-4e67-84a2-7d1ab1b73c3f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:41.618107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 157/318] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:29:41 2022] GET /identity/v3/role_assignments?user.id=2895d98f8f8846379d71a0f4f0740b5f&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=a3a50df9b07a42c4b1361e06ea200403 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:42.137863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a595-4e40-4785-91b2-0ef05dfdb4a4 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.138169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a595-4e40-4785-91b2-0ef05dfdb4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.138429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a595-4e40-4785-91b2-0ef05dfdb4a4 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.139968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 162/319] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:42.145006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528013-92b3-4cac-8294-08116a5b9df9 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.145466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528013-92b3-4cac-8294-08116a5b9df9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.145542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528013-92b3-4cac-8294-08116a5b9df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.163014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-62528013-92b3-4cac-8294-08116a5b9df9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:42.164207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-62528013-92b3-4cac-8294-08116a5b9df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:42.232822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 158/320] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:42.254413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a595-4e40-4785-91b2-0ef05dfdb4a4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:42.356693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a595-4e40-4785-91b2-0ef05dfdb4a4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:42.390148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02593f-5e21-49ea-b03e-4a526acd95d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:42.390949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02593f-5e21-49ea-b03e-4a526acd95d6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.391238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02593f-5e21-49ea-b03e-4a526acd95d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.391520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02593f-5e21-49ea-b03e-4a526acd95d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.402255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02593f-5e21-49ea-b03e-4a526acd95d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-bf02593f-5e21-49ea-b03e-4a526acd95d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:42.410312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:42.411185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:42.411967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:42.416521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 163/321] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:42.417952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62528013-92b3-4cac-8294-08116a5b9df9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:42.454611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc54bc-7943-4a07-a673-c06e869d6fed None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.454900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc54bc-7943-4a07-a673-c06e869d6fed None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.455176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc54bc-7943-4a07-a673-c06e869d6fed None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.456743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 164/322] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:42.462202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36599-95d4-475d-b2e0-eeae41f0f65d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.462441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36599-95d4-475d-b2e0-eeae41f0f65d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.462713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd36599-95d4-475d-b2e0-eeae41f0f65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.466810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:42.470600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:42.481538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bdd36599-95d4-475d-b2e0-eeae41f0f65d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:42.483320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bdd36599-95d4-475d-b2e0-eeae41f0f65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:42.528208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62528013-92b3-4cac-8294-08116a5b9df9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:42.566337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 165/323] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:42.566521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:42.567147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.567526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.567806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.568582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:42.568889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:42.570255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:42.580121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 159/324] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:42.585610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36599-95d4-475d-b2e0-eeae41f0f65d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:42.590066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:42.693013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd36599-95d4-475d-b2e0-eeae41f0f65d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:42.703284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa8268-8eaa-493e-b929-785c81b84828 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:42.714452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:42.726986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0a8f6-e442-472d-a9ad-7b69c80c466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:42.727903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a8f6-e442-472d-a9ad-7b69c80c466e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.728187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a8f6-e442-472d-a9ad-7b69c80c466e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.728497 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a8f6-e442-472d-a9ad-7b69c80c466e None admin] PATH_INFO: `/v3/users/1d993b1406894111889972ce58482a68` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.729714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0a8f6-e442-472d-a9ad-7b69c80c466e None admin] RBAC: Authorizing `identity:get_user(user_id=1d993b1406894111889972ce58482a68)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:42.731157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0a8f6-e442-472d-a9ad-7b69c80c466e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:42.732822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 166/325] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:42 2022] GET /identity/v3/users/1d993b1406894111889972ce58482a68 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:42.733975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2452c0-aee7-4311-bc20-0a62ea56955a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.734264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2452c0-aee7-4311-bc20-0a62ea56955a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.734540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2452c0-aee7-4311-bc20-0a62ea56955a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.735766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 167/326] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:42.738436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2452c0-aee7-4311-bc20-0a62ea56955a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:42.739914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc075ec-1136-4b17-af64-84f41cd2f59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:42.740742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc075ec-1136-4b17-af64-84f41cd2f59f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.741157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:42.741436 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc075ec-1136-4b17-af64-84f41cd2f59f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.742556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc075ec-1136-4b17-af64-84f41cd2f59f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.743763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc075ec-1136-4b17-af64-84f41cd2f59f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:42.745231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc075ec-1136-4b17-af64-84f41cd2f59f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:42.746837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 160/327] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:29:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:42.748104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b827-a4ea-4006-b54b-77226668be14 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.748393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b827-a4ea-4006-b54b-77226668be14 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.748696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b827-a4ea-4006-b54b-77226668be14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.767139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-7f82b827-a4ea-4006-b54b-77226668be14 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:42.768428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7f82b827-a4ea-4006-b54b-77226668be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:42.833024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 161/328] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:42.833806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b827-a4ea-4006-b54b-77226668be14 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:42.845143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2452c0-aee7-4311-bc20-0a62ea56955a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:42.877902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:42.878760 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.879042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.879314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] PATH_INFO: `/v3/system/users/1d993b1406894111889972ce58482a68/roles/a3a50df9b07a42c4b1361e06ea200403` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.880801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1d993b1406894111889972ce58482a68, role_id=a3a50df9b07a42c4b1361e06ea200403)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:42.882166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:42.928635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b827-a4ea-4006-b54b-77226668be14 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:42.936163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:42.941731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 168/329] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Jul 20 17:29:42 2022] PUT /identity/v3/system/users/1d993b1406894111889972ce58482a68/roles/a3a50df9b07a42c4b1361e06ea200403 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:42.942558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:42.970681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:42.971895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:42.972303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:42.972519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] PATH_INFO: `/v3/groups` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:42.973195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:42.974724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:43.021464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:43.047005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 162/330] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Jul 20 17:29:42 2022] POST /identity/v3/groups => generated 226 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:29:43.062459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d921-348f-4bca-a507-1b250bc110ba None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:43.107721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:43.107977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:43.108259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:43.108756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:43.123154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:43.130992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:43.131963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:43.132792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 169/331] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:43.140190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:43.246952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8eb3-dc56-436b-aad1-68841b996c63 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:43.282223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:43.282770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:43.283054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:43.283329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:43.284075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:43.284381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:43.285805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:43.295795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 163/332] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:43.301293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:43.411570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbebd01-c0b0-477f-8843-d66014f698cf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:43.439851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:43.449458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fbf1d-b478-479e-a9f2-9171c7b78958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:43.450336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fbf1d-b478-479e-a9f2-9171c7b78958 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:43.450618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fbf1d-b478-479e-a9f2-9171c7b78958 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:43.450890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fbf1d-b478-479e-a9f2-9171c7b78958 None admin] PATH_INFO: `/v3/users/817126fb17764b4e98996d449e861810` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:43.452097 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fbf1d-b478-479e-a9f2-9171c7b78958 None admin] RBAC: Authorizing `identity:get_user(user_id=817126fb17764b4e98996d449e861810)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:43.453579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fbf1d-b478-479e-a9f2-9171c7b78958 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:43.455540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 170/333] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:43 2022] GET /identity/v3/users/817126fb17764b4e98996d449e861810 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:43.462012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:43.492844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:43.566640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd305e-f1fc-4eff-ba9f-2e29983230ef None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:43.601925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10313e-bf03-4528-89b9-7d53c4b1a840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:43.603127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10313e-bf03-4528-89b9-7d53c4b1a840 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:43.603407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10313e-bf03-4528-89b9-7d53c4b1a840 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:43.603683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10313e-bf03-4528-89b9-7d53c4b1a840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:43.604550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10313e-bf03-4528-89b9-7d53c4b1a840 None admin] RBAC: Adding query filter params (role.id=295c91f6f2554a0ea6f954d898f9725c, user.id=817126fb17764b4e98996d449e861810, scope.system=all) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:43.604864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10313e-bf03-4528-89b9-7d53c4b1a840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:43.606291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10313e-bf03-4528-89b9-7d53c4b1a840 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:43.619142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 164/334] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jul 20 17:29:43 2022] GET /identity/v3/role_assignments?user.id=817126fb17764b4e98996d449e861810&scope.system=all&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 510 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:45.139820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d8550-1bcf-47e8-af90-9bd0cb3bf517 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.140329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d8550-1bcf-47e8-af90-9bd0cb3bf517 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.140402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d8550-1bcf-47e8-af90-9bd0cb3bf517 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.141916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 171/335] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:45.147488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffee7a-4d50-43bc-8e06-966722d06f9b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.147911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffee7a-4d50-43bc-8e06-966722d06f9b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.148088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffee7a-4d50-43bc-8e06-966722d06f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.166894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3fffee7a-4d50-43bc-8e06-966722d06f9b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:45.168073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3fffee7a-4d50-43bc-8e06-966722d06f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:45.246755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 165/336] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:45.280793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d8550-1bcf-47e8-af90-9bd0cb3bf517 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:45.404222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d8550-1bcf-47e8-af90-9bd0cb3bf517 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:45.456317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:45.457152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.457439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.457720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.477352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:45.485172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:45.487251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:45.489235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 172/337] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:45.497584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffee7a-4d50-43bc-8e06-966722d06f9b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:45.534206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:45.663180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffee7a-4d50-43bc-8e06-966722d06f9b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:45.711870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79665098-ae99-47ef-98fb-9de338407bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:45.712571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79665098-ae99-47ef-98fb-9de338407bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.712854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79665098-ae99-47ef-98fb-9de338407bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.713129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79665098-ae99-47ef-98fb-9de338407bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.713886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79665098-ae99-47ef-98fb-9de338407bd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:45.714196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79665098-ae99-47ef-98fb-9de338407bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:45.715572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79665098-ae99-47ef-98fb-9de338407bd3 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:45.727568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 166/338] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:45.733072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:45.738307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b93d1-518a-48fb-b0fd-203cfa823b33 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.738595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b93d1-518a-48fb-b0fd-203cfa823b33 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.738872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b93d1-518a-48fb-b0fd-203cfa823b33 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.740525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 167/339] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:45.746825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c6d4-b7df-46c0-aff8-b07f5d53a3c8 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.747108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c6d4-b7df-46c0-aff8-b07f5d53a3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.747457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c6d4-b7df-46c0-aff8-b07f5d53a3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.770849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-2346c6d4-b7df-46c0-aff8-b07f5d53a3c8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:45.772030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2346c6d4-b7df-46c0-aff8-b07f5d53a3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:45.842911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:45.895248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 168/340] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:45.896730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3c2-b7b4-4675-a986-d9ac72b0a1eb None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.897027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3c2-b7b4-4675-a986-d9ac72b0a1eb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.897312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3c2-b7b4-4675-a986-d9ac72b0a1eb None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.898870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 169/341] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:45.905691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606950d5-318c-48a5-9803-fcf53afc77b4 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.905981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606950d5-318c-48a5-9803-fcf53afc77b4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.906265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606950d5-318c-48a5-9803-fcf53afc77b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.918428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059532e-9a41-4057-b06f-da49adb2898b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:45.945491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-606950d5-318c-48a5-9803-fcf53afc77b4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:45.947242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-606950d5-318c-48a5-9803-fcf53afc77b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:45.949468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:45.966998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:45.967883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:45.968164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:45.968457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] PATH_INFO: `/v3/users/1d993b1406894111889972ce58482a68` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:45.971286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] RBAC: Authorizing `identity:get_user(user_id=1d993b1406894111889972ce58482a68)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:45.973206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:45.974668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 173/342] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:29:45 2022] GET /identity/v3/users/1d993b1406894111889972ce58482a68 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:45.975327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:46.057531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 170/343] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:46.058667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606950d5-318c-48a5-9803-fcf53afc77b4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:46.140646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1462d1-7dbe-40bd-97e9-c5e57aa1e800 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:46.179264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:46.201838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:46.203128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:46.204980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:46.205712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:46.230378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:46.242187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:46.243674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:46.244501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:46.250538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 174/344] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:46.252759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:46.285011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606950d5-318c-48a5-9803-fcf53afc77b4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:46.386406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:46.387260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:46.387640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:46.388025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:46.389000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] RBAC: Adding query filter params (role.id=a3a50df9b07a42c4b1361e06ea200403, user.id=1d993b1406894111889972ce58482a68, scope.system=all) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:46.389598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:46.391030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:46.397620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:46.417960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 171/345] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Jul 20 17:29:46 2022] GET /identity/v3/role_assignments?user.id=1d993b1406894111889972ce58482a68&scope.system=all&role.id=a3a50df9b07a42c4b1361e06ea200403 => generated 510 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:46.418971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:46.441960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0783-cf7a-4529-a279-5132227bb1fc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:46.491777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74dc6bd-1b53-471c-9f82-282315dc5c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:46.493505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dc6bd-1b53-471c-9f82-282315dc5c46 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:46.494864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dc6bd-1b53-471c-9f82-282315dc5c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:46.495303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74dc6bd-1b53-471c-9f82-282315dc5c46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:46.524885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74dc6bd-1b53-471c-9f82-282315dc5c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-a74dc6bd-1b53-471c-9f82-282315dc5c46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:46.543965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:46.546249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:46.547080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:46.549575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 175/346] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:46.556157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2617d-fd25-4d47-81fc-72ff01b82f32 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:46.556941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2617d-fd25-4d47-81fc-72ff01b82f32 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:46.557365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2617d-fd25-4d47-81fc-72ff01b82f32 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:46.559479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 176/347] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:46.564819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858842-3dd6-4acb-9cd6-4444fde1b06b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:46.566583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858842-3dd6-4acb-9cd6-4444fde1b06b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:46.567017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858842-3dd6-4acb-9cd6-4444fde1b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:46.605239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cd858842-3dd6-4acb-9cd6-4444fde1b06b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:46.606830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cd858842-3dd6-4acb-9cd6-4444fde1b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:46.614920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2147a75-e5e7-480a-aad7-d02aea761109 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:46.688935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:46.688935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:46.688935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:46.689558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:46.690582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:46.691033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:46.693297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:46.709482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 172/348] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:46.713346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:46.718124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:46.787307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 177/349] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:46.788870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858842-3dd6-4acb-9cd6-4444fde1b06b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:46.877926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:46.969165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858842-3dd6-4acb-9cd6-4444fde1b06b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:46.983948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3877a4-1d8b-4bf3-86a2-9c52b7c88bed None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.010351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.011014 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.011302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.011581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.037802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.038657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.039143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.039434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.040878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:47.041190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:47.042672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:47.052734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 173/350] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:47.058761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:47.060216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:47.084045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:47.085603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:47.205487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc62a2-95be-4587-bff0-c4bb9f862bea None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.229905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:47.233328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 178/351] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:47.234268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:47.246134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.247213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.247677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.250308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.263595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:47.274488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:47.275374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:47.276168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:47.278941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 174/352] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:47.279923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:47.326420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:47.360236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbb349-1e91-439d-8714-640fe9401c20 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.399358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.400359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.401038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.401538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.418388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1715c-ff55-46b3-b3bc-5f31a74b50e8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.467124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.468361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.468877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.469374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.471132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:47.472972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:47.475445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 175/353] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:47.476411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:47.484282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:47.485730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:47.541658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:47.566612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:47.603547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 179/354] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:47.604761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:47.615191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffc3ec-4201-4198-968d-8e0d65f22843 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.656448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1781ba-5fb2-4bb7-aa4d-bef47cf76c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.657395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1781ba-5fb2-4bb7-aa4d-bef47cf76c24 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.657680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1781ba-5fb2-4bb7-aa4d-bef47cf76c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.657955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1781ba-5fb2-4bb7-aa4d-bef47cf76c24 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.658700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1781ba-5fb2-4bb7-aa4d-bef47cf76c24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:47.659006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1781ba-5fb2-4bb7-aa4d-bef47cf76c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:47.660366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1781ba-5fb2-4bb7-aa4d-bef47cf76c24 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:47.674124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 176/355] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:47.675566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fece12-5297-4cff-8450-24013822909e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.675859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fece12-5297-4cff-8450-24013822909e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.676141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fece12-5297-4cff-8450-24013822909e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.678785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 177/356] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:47.679559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fece12-5297-4cff-8450-24013822909e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:47.722847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f4eaf-1fd9-47b4-94ef-b528ccd745b8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.767764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.768688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.768975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.769252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.778100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:47.779557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:47.781462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 180/357] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:47.782350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:47.810410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fece12-5297-4cff-8450-24013822909e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.823825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:47.825051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:47.850503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee3d7-0ea1-4a7f-820d-cbd095cbff45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.851817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee3d7-0ea1-4a7f-820d-cbd095cbff45 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.851817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee3d7-0ea1-4a7f-820d-cbd095cbff45 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.852681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee3d7-0ea1-4a7f-820d-cbd095cbff45 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.854628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee3d7-0ea1-4a7f-820d-cbd095cbff45 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:47.855103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee3d7-0ea1-4a7f-820d-cbd095cbff45 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:47.858691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 178/358] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:47.858691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ddedf-c3c8-48b7-99fe-84a5b551b8fa None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.858691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ddedf-c3c8-48b7-99fe-84a5b551b8fa None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.858691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ddedf-c3c8-48b7-99fe-84a5b551b8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.879924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-328ddedf-c3c8-48b7-99fe-84a5b551b8fa None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:47.881160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-328ddedf-c3c8-48b7-99fe-84a5b551b8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:47.899551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd8e96-10cb-4134-81d9-54ff19dfbd50 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:47.941952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:47.942615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:47.942897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:47.943235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:47.968084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 179/359] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:47.968996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ddedf-c3c8-48b7-99fe-84a5b551b8fa None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.019598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:48.021151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:48.060127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.087365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ddedf-c3c8-48b7-99fe-84a5b551b8fa None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.098514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.130342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cde2b-3950-4d60-bfe4-a2e1fda8a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:48.131934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cde2b-3950-4d60-bfe4-a2e1fda8a8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.131934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cde2b-3950-4d60-bfe4-a2e1fda8a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.131934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cde2b-3950-4d60-bfe4-a2e1fda8a8a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.132844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cde2b-3950-4d60-bfe4-a2e1fda8a8a4 None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:48.133217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cde2b-3950-4d60-bfe4-a2e1fda8a8a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:48.134641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cde2b-3950-4d60-bfe4-a2e1fda8a8a4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:48.146131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 180/360] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:48.146131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765202-b927-4b55-bfed-59f33712afe6 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.146131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765202-b927-4b55-bfed-59f33712afe6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.146131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765202-b927-4b55-bfed-59f33712afe6 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.146131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 181/361] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:48.146564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf765202-b927-4b55-bfed-59f33712afe6 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.150938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 181/362] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:48.151816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.291674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf765202-b927-4b55-bfed-59f33712afe6 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.307625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0fd5b-7b30-427f-a461-fae4cee18d70 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.337179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf59520-9f67-4e6f-b8b8-4444017f2dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:48.338107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf59520-9f67-4e6f-b8b8-4444017f2dbc None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.338392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf59520-9f67-4e6f-b8b8-4444017f2dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.338663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf59520-9f67-4e6f-b8b8-4444017f2dbc None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.341321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf59520-9f67-4e6f-b8b8-4444017f2dbc None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:48.341436 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf59520-9f67-4e6f-b8b8-4444017f2dbc None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:48.343178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 182/363] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:48.344191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dcf88-e484-4fcb-8182-4c3b6cc486fb None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.346661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dcf88-e484-4fcb-8182-4c3b6cc486fb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.346661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dcf88-e484-4fcb-8182-4c3b6cc486fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.360294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:48.361376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.361828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.362262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.363919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0e4dcf88-e484-4fcb-8182-4c3b6cc486fb None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:48.365137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0e4dcf88-e484-4fcb-8182-4c3b6cc486fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:48.367263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.379141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:48.387527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:48.389368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:48.390224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:48.391793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 182/364] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:48.397192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.408178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.445504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 183/365] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:48.446999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dcf88-e484-4fcb-8182-4c3b6cc486fb None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.532485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da661f-7812-408b-92c6-ce07d4ccba56 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.577422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dcf88-e484-4fcb-8182-4c3b6cc486fb None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.582395 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:48.583146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.583426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.583704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.588885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:48.588885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:48.593504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 183/366] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:48.594373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.632042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:48.632787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.633068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.633899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.634728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:48.635036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:48.636666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:48.644244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 184/367] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=4182f79f5cda459497c049f3f07a1713 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:48.646406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.690199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.690448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.742122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e93d6-91b7-4fff-b8cb-9850e85ed85c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.776617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdc285-40fa-4d55-8e4f-6d3ac5ebe953 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.790580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:48.791712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.792097 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.792493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.794122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:48.794581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:48.796398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:48.809412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 184/368] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:48.810420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.821270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:48.822078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:48.822420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:48.822810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:48.836180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:48.845710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:48.847840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 185/369] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:48.849814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:48.943478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fdfeb-9339-4514-8bbe-af8d5ac988f0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.972874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.983267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51d4af-b39d-4800-bd55-f95e0fe50974 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:48.988668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:48.999398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:49.000400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:49.000737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:49.001233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:49.002450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:49.005274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:49.007205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 185/370] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:49.008020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:49.030217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:49.030914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:49.031505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:49.032046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:49.127902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:49.129425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:49.147869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd792aa-3419-4db1-abd1-f15e7c7ee72e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:49.193720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:49.195311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:49.195941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:49.196461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:49.197712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:49.198589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:49.200533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:49.213759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 186/371] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:49.214949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:49.245193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:49.255091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 186/372] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:49.256288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:49.281631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:49.333064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf6701-e9e8-4422-b598-938482022917 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:49.374098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:49.374799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:49.375162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:49.375512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:49.376417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=4f8537768c1b489d865a39e704eb9b79) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:49.376859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:49.378440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:49.388470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 187/373] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:49 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:49.388470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:49.416916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a2ffa-e17f-4099-8087-1d1d5c9e7dec None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:49.475735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:49.476401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:49.476697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:49.477512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:49.563322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbfe02-1c18-45f4-8ca9-1381abac1e2d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:49.605626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:49.624776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:49.630673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:49.632130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:49.638200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:49.639108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:49.639394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:49.639670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:49.645227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:49.647430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:49.650206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 188/374] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:49 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:49.655208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:49.750921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 187/375] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:49.755769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:49.787484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29ac3-64c2-4235-99ec-deab5cb328ac None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:49.862340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:49.863206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:49.863496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:49.863757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:49.865178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:49.866620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:49.873692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 189/376] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:49 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:49.874546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:49.941070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:49.951498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bd060-5fff-47dd-9a41-b58d65bc9482 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:49.956683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:50.014685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f16e2b-6fcb-4b86-be32-87888343a2de None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:50.046498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:50.047394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:50.047679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:50.047960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] PATH_INFO: `/v3/projects/3ddf29534d9849ed9221c0b30b65b542` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:50.050791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] RBAC: Authorizing `identity:get_project(project_id=3ddf29534d9849ed9221c0b30b65b542)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:50.052286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:50.054126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 188/377] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:49 2022] GET /identity/v3/projects/3ddf29534d9849ed9221c0b30b65b542 => generated 344 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:50.060147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:50.067454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:50.068124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:50.068407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:50.068711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:50.069974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=4f8537768c1b489d865a39e704eb9b79) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:50.070294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:50.071877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:50.082348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:49 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=4182f79f5cda459497c049f3f07a1713 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:50.213729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428ecb-bd90-424d-9f35-83827f62816e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:50.259215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89637acc-52fe-4884-919b-cbbcda472180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:50.260077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637acc-52fe-4884-919b-cbbcda472180 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:50.260360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637acc-52fe-4884-919b-cbbcda472180 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:50.260659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637acc-52fe-4884-919b-cbbcda472180 None admin] PATH_INFO: `/v3/groups/9b763f0ed8834c66bad11d23e7b2fa4b` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:50.267691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89637acc-52fe-4884-919b-cbbcda472180 None admin] RBAC: Authorizing `identity:get_group(group_id=9b763f0ed8834c66bad11d23e7b2fa4b)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:50.269171 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89637acc-52fe-4884-919b-cbbcda472180 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:50.270856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 189/379] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:50 2022] GET /identity/v3/groups/9b763f0ed8834c66bad11d23e7b2fa4b => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:50.276837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:50.386370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987c5f4-886a-493b-b2e9-cfffad8262c6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:50.407268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:50.425239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419d057-e92f-45cd-8b29-337b2bc45bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:50.425915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419d057-e92f-45cd-8b29-337b2bc45bea None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:50.426205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419d057-e92f-45cd-8b29-337b2bc45bea None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:50.426490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419d057-e92f-45cd-8b29-337b2bc45bea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:50.427335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419d057-e92f-45cd-8b29-337b2bc45bea None admin] RBAC: Adding query filter params (group.id=9b763f0ed8834c66bad11d23e7b2fa4b, role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=3ddf29534d9849ed9221c0b30b65b542) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:50.427656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419d057-e92f-45cd-8b29-337b2bc45bea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:50.429153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419d057-e92f-45cd-8b29-337b2bc45bea None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:50.435602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 191/380] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:50 2022] GET /identity/v3/role_assignments?group.id=9b763f0ed8834c66bad11d23e7b2fa4b&scope.project.id=3ddf29534d9849ed9221c0b30b65b542&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:50.684067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a9a5-abb0-40dd-a66e-6bf47a9e497d None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:50.684647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a9a5-abb0-40dd-a66e-6bf47a9e497d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:50.684647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09a9a5-abb0-40dd-a66e-6bf47a9e497d None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:50.686404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 190/381] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:50.694274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97fe34-8157-4d72-933f-3881a984c851 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:50.694555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97fe34-8157-4d72-933f-3881a984c851 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:50.694840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97fe34-8157-4d72-933f-3881a984c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:50.717651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4a97fe34-8157-4d72-933f-3881a984c851 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:50.718891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4a97fe34-8157-4d72-933f-3881a984c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:50.803923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 192/382] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:50.826746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09a9a5-abb0-40dd-a66e-6bf47a9e497d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:50.861672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:50.946124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09a9a5-abb0-40dd-a66e-6bf47a9e497d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:50.990551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc3d77-a43a-44ee-b1bb-d778f054a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:50.991353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc3d77-a43a-44ee-b1bb-d778f054a0dd None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:50.991640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc3d77-a43a-44ee-b1bb-d778f054a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:50.991921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc3d77-a43a-44ee-b1bb-d778f054a0dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.005563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc3d77-a43a-44ee-b1bb-d778f054a0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e8dc3d77-a43a-44ee-b1bb-d778f054a0dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:51.015395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:51.016364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:51.017268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 191/383] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:51.022882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97fe34-8157-4d72-933f-3881a984c851 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.108329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0813f-bd1d-467e-b305-bd2036503547 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.108639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0813f-bd1d-467e-b305-bd2036503547 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.108921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0813f-bd1d-467e-b305-bd2036503547 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.111373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 192/384] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:29:51.118061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0dfac-c7d2-4d3c-bfc5-7bf4ff72a0ef None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.118354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0dfac-c7d2-4d3c-bfc5-7bf4ff72a0ef None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.118639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0dfac-c7d2-4d3c-bfc5-7bf4ff72a0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.140156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d4f0dfac-c7d2-4d3c-bfc5-7bf4ff72a0ef None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:51.141864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d4f0dfac-c7d2-4d3c-bfc5-7bf4ff72a0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:51.166235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97fe34-8157-4d72-933f-3881a984c851 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:51.193679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:51.222478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:51.223075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.223362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.223641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.224390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:51.224752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:51.226145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:51.236963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 193/385] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:51.241832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 193/386] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:51.242755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.282198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0dfac-c7d2-4d3c-bfc5-7bf4ff72a0ef None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.309520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:51.351642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238212cf-0d6c-4ff3-a4c4-86e2c8b62b4a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:51.392267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:51.393424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.394130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.394417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.396159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:51.398212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:51.400749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 194/387] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:51 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:51.408693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.448335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0dfac-c7d2-4d3c-bfc5-7bf4ff72a0ef None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:51.464527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:51.496191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:51.497017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.497354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.497639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.512594 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:51.521062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:51.521945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:51.522862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 194/388] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:51.528886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.531257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c269c0-9e15-4c50-9afb-44229b8bd21c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:51.599617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:51.600820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.601109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.601388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.603921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:51.605651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:51.608627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 195/389] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:51 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:51.616467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.684719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:51.712796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f386c73-77b9-42cb-b9b9-9cbd6f38a179 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:51.756447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:51.759088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.760858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.761327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.763055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:51.764649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:51.766440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:51.786438 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 195/390] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:51.806557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.846526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cffe97-9cbb-4a05-abfb-0c95707bc8ca None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:51.875173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:51.924340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d8a15-d831-4b1f-8cb3-3188e1d3ad93 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:51.937167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada77f0-ddff-455a-8e80-6d5630447a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:51.938050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada77f0-ddff-455a-8e80-6d5630447a49 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.938350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada77f0-ddff-455a-8e80-6d5630447a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.938592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada77f0-ddff-455a-8e80-6d5630447a49 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/groups/794f0ffb3816401baac5bbbd399580e8/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.945205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada77f0-ddff-455a-8e80-6d5630447a49 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a43fae56142df9ddb88b6d5c798fb, group_id=794f0ffb3816401baac5bbbd399580e8, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:51.947208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada77f0-ddff-455a-8e80-6d5630447a49 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:51.964591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:51.967033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:51.967033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:51.967033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:51.968044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:51.969713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:51.971916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 196/391] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:51 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:51.977791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:51.993909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:52.059021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 196/392] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jul 20 17:29:51 2022] PUT /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/groups/794f0ffb3816401baac5bbbd399580e8/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:52.060407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f35c8-e84e-4975-b020-272ec80bb4ae None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.060773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f35c8-e84e-4975-b020-272ec80bb4ae None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.061056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f35c8-e84e-4975-b020-272ec80bb4ae None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.067458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 197/393] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:52.070146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180051ea-241a-4d91-af7e-467b8d8f7edf None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.070516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180051ea-241a-4d91-af7e-467b8d8f7edf None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.071544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180051ea-241a-4d91-af7e-467b8d8f7edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.099868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-180051ea-241a-4d91-af7e-467b8d8f7edf None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:52.107483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-180051ea-241a-4d91-af7e-467b8d8f7edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:52.135045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2210c06-559b-4298-b7aa-d26fcf741c55 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:52.198908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 198/394] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:52.220801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180051ea-241a-4d91-af7e-467b8d8f7edf None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:52.253219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:52.254110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.254400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.254683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.256019 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:52.257510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:52.259628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 197/395] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:51 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:52.270675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:52.334382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180051ea-241a-4d91-af7e-467b8d8f7edf None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:52.337016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:52.376509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:52.377726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.378243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.378713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.390816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:52.400243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:52.401783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:52.402774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:52.404136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:52.409925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 199/396] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:52.412052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:52.424743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14a235-a6b1-41f9-a680-25bda7a28c82 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:52.494150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:52.495304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.495746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.496166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/groups/794f0ffb3816401baac5bbbd399580e8/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.498535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a43fae56142df9ddb88b6d5c798fb, group_id=794f0ffb3816401baac5bbbd399580e8, role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:52.500166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:52.556231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd170-a5f4-4be7-bae0-bdc261c0ed23 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:52.577835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 198/397] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jul 20 17:29:52 2022] PUT /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb/groups/794f0ffb3816401baac5bbbd399580e8/roles/4182f79f5cda459497c049f3f07a1713 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:29:52.617581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1db11-896c-4ce8-bbef-2746a4018f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:52.619761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1db11-896c-4ce8-bbef-2746a4018f5f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.620600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1db11-896c-4ce8-bbef-2746a4018f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.621340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1db11-896c-4ce8-bbef-2746a4018f5f None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.623104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1db11-896c-4ce8-bbef-2746a4018f5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:52.624005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1db11-896c-4ce8-bbef-2746a4018f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:52.627156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1db11-896c-4ce8-bbef-2746a4018f5f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:52.646475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 200/398] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:52.652873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:52.690493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d62b3-d5ee-418a-bd0a-e637fdc9a7fb None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.691009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d62b3-d5ee-418a-bd0a-e637fdc9a7fb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.691445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d62b3-d5ee-418a-bd0a-e637fdc9a7fb None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.693088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 201/399] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:52.699242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262bed1-c59a-4b0e-8d7f-64bf753f716c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.699736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262bed1-c59a-4b0e-8d7f-64bf753f716c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.700208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262bed1-c59a-4b0e-8d7f-64bf753f716c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.723776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-9262bed1-c59a-4b0e-8d7f-64bf753f716c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:52.731470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9262bed1-c59a-4b0e-8d7f-64bf753f716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:52.754096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:52.810459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d6cb8-760f-4940-b5e4-657dec23f685 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:52.819354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 202/400] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:29:52.849761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262bed1-c59a-4b0e-8d7f-64bf753f716c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:52.867063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:52.868504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:52.868782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:52.869121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:52.873161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:52.874630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:52.879688 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 199/401] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:52 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:52.897477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:52.906744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:52.969521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262bed1-c59a-4b0e-8d7f-64bf753f716c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.013675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.015578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.016254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.018174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.040627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:53.057613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:53.058783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:53.059758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:53.066471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 203/402] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:53.073633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:53.091450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:53.178603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df066b6f-e9ab-4170-a876-89e15976f6c7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.192087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5150-44f4-475c-a9de-f3edd9b065ca None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.239060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.239869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.240161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.240452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.241978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:53.242301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:53.245478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:53.258816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.259903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.260191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.262257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 204/403] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:53.262526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.263821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:53.265330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:53.267815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:53.271754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 200/404] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:52 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:53.274632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:53.387974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:53.388180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:53.411528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225570-82b9-4bdb-82a4-a87fb91a06d7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.413611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39deb3c-1dc3-4a7d-8e24-eb7eca537dc9 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.448924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1a01-5d49-40d9-bb8a-0bbad60acfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.449704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1a01-5d49-40d9-bb8a-0bbad60acfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.449987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1a01-5d49-40d9-bb8a-0bbad60acfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.450262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1a01-5d49-40d9-bb8a-0bbad60acfc1 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.451913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1a01-5d49-40d9-bb8a-0bbad60acfc1 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:53.454265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1a01-5d49-40d9-bb8a-0bbad60acfc1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:53.460401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 205/405] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:53 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:53.460401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94803-b800-4659-9915-213c68eb790a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.460401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94803-b800-4659-9915-213c68eb790a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.460401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94803-b800-4659-9915-213c68eb790a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.460401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 206/406] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:53.462512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf4273-d34d-429a-a7f8-f9c677ca4193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.462724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf4273-d34d-429a-a7f8-f9c677ca4193 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.463997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf4273-d34d-429a-a7f8-f9c677ca4193 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.463997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf4273-d34d-429a-a7f8-f9c677ca4193 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79/groups/794f0ffb3816401baac5bbbd399580e8/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.465214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf4273-d34d-429a-a7f8-f9c677ca4193 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f8537768c1b489d865a39e704eb9b79, group_id=794f0ffb3816401baac5bbbd399580e8, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:53.466971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf4273-d34d-429a-a7f8-f9c677ca4193 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:53.469628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b94803-b800-4659-9915-213c68eb790a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:53.573092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 201/407] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jul 20 17:29:53 2022] PUT /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79/groups/794f0ffb3816401baac5bbbd399580e8/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:53.574823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cd6b8-2d05-4b87-8954-b6d2dda74260 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.575119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cd6b8-2d05-4b87-8954-b6d2dda74260 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.575398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cd6b8-2d05-4b87-8954-b6d2dda74260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.611941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f40cd6b8-2d05-4b87-8954-b6d2dda74260 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:53.613198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f40cd6b8-2d05-4b87-8954-b6d2dda74260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:53.617608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b94803-b800-4659-9915-213c68eb790a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.692743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.693632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.693915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.694191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.695461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:53.697000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:53.698981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 207/408] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:53 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:53.706092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:53.729311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:53.753825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 202/409] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:53.783933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cd6b8-2d05-4b87-8954-b6d2dda74260 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:53.847296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace32b0-2103-4089-b245-a00e36227891 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.884080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:53.885224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.885951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.886350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.886720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79/groups/794f0ffb3816401baac5bbbd399580e8/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.889608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] RBAC: Authorizing `identity:create_grant(project_id=4f8537768c1b489d865a39e704eb9b79, group_id=794f0ffb3816401baac5bbbd399580e8, role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:53.891189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:53.933875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cd6b8-2d05-4b87-8954-b6d2dda74260 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:53.976008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 208/410] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jul 20 17:29:53 2022] PUT /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79/groups/794f0ffb3816401baac5bbbd399580e8/roles/4182f79f5cda459497c049f3f07a1713 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:29:53.977549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:53.978511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:53.978903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:53.979274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:53.993815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:54.002423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:54.003381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:54.004178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 203/411] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:54.011398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:54.137165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:54.179900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfbc53-2531-4343-9179-97823d36414b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:54.230061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:54.230658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.230950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.231230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.231990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:54.232303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:54.233711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:54.250981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 209/412] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:54.256713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:54.391541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037f44f-5037-406e-a47a-e6724c810481 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:54.413328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:54.427215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5665f54-b3db-4b83-9586-95369a169433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:54.428079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5665f54-b3db-4b83-9586-95369a169433 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.428360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5665f54-b3db-4b83-9586-95369a169433 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.428669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5665f54-b3db-4b83-9586-95369a169433 None admin] PATH_INFO: `/v3/groups/9b763f0ed8834c66bad11d23e7b2fa4b` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.429848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5665f54-b3db-4b83-9586-95369a169433 None admin] RBAC: Authorizing `identity:get_group(group_id=9b763f0ed8834c66bad11d23e7b2fa4b)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:54.431938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5665f54-b3db-4b83-9586-95369a169433 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:54.438104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 204/413] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:54 2022] GET /identity/v3/groups/9b763f0ed8834c66bad11d23e7b2fa4b => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:54.445587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:54.515230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57b3d0-1663-400c-9637-44f92287e275 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.515540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57b3d0-1663-400c-9637-44f92287e275 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.515705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57b3d0-1663-400c-9637-44f92287e275 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.517365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 205/414] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:54.522339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eb9df-7973-4f8a-8197-221caa6d35b6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.522627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eb9df-7973-4f8a-8197-221caa6d35b6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.522906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eb9df-7973-4f8a-8197-221caa6d35b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.548380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-da0eb9df-7973-4f8a-8197-221caa6d35b6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:54.549591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-da0eb9df-7973-4f8a-8197-221caa6d35b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:54.609233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ad592-a077-4e4b-ab0e-56931d9689c6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:54.653546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:54.654981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.656094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.656771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] PATH_INFO: `/v3/projects/3ddf29534d9849ed9221c0b30b65b542` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.659532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] RBAC: Authorizing `identity:get_project(project_id=3ddf29534d9849ed9221c0b30b65b542)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:54.661960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:54.665560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 210/415] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:54 2022] GET /identity/v3/projects/3ddf29534d9849ed9221c0b30b65b542 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:54.671245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:54.676747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:54.691232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 206/416] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:54.714672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0eb9df-7973-4f8a-8197-221caa6d35b6 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:54.866280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800497-8402-432a-ae35-c91c6a342c50 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:54.904294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0eb9df-7973-4f8a-8197-221caa6d35b6 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:54.927133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:54.928351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa52b8-9d63-479f-9825-8f538145ee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:54.929338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa52b8-9d63-479f-9825-8f538145ee9e None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.929686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa52b8-9d63-479f-9825-8f538145ee9e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.930080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa52b8-9d63-479f-9825-8f538145ee9e None admin] PATH_INFO: `/v3/projects/3ddf29534d9849ed9221c0b30b65b542/groups/9b763f0ed8834c66bad11d23e7b2fa4b/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.932850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa52b8-9d63-479f-9825-8f538145ee9e None admin] RBAC: Authorizing `identity:create_grant(project_id=3ddf29534d9849ed9221c0b30b65b542, group_id=9b763f0ed8834c66bad11d23e7b2fa4b, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:54.935128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa52b8-9d63-479f-9825-8f538145ee9e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:54.944605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16aef48-c0c6-4fdc-9dad-b453be6f71b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:54.945430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aef48-c0c6-4fdc-9dad-b453be6f71b9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.946219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aef48-c0c6-4fdc-9dad-b453be6f71b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.946494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16aef48-c0c6-4fdc-9dad-b453be6f71b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.956366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16aef48-c0c6-4fdc-9dad-b453be6f71b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d16aef48-c0c6-4fdc-9dad-b453be6f71b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:54.964766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:54.971554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 207/417] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:54.972392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdafbf-4dce-4644-8716-5d74a79422ad None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:54.973145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdafbf-4dce-4644-8716-5d74a79422ad None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:54.973145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdafbf-4dce-4644-8716-5d74a79422ad None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:54.973834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 208/418] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:54.974367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdafbf-4dce-4644-8716-5d74a79422ad None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:55.032270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 211/419] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Jul 20 17:29:54 2022] PUT /identity/v3/projects/3ddf29534d9849ed9221c0b30b65b542/groups/9b763f0ed8834c66bad11d23e7b2fa4b/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:29:55.032270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2763cf-a900-4d7c-80f1-e027ff60eee8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:55.032270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2763cf-a900-4d7c-80f1-e027ff60eee8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:55.032270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2763cf-a900-4d7c-80f1-e027ff60eee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:55.068600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-aa2763cf-a900-4d7c-80f1-e027ff60eee8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:55.070058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-aa2763cf-a900-4d7c-80f1-e027ff60eee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:55.151996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdafbf-4dce-4644-8716-5d74a79422ad None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:55.207584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:55.225551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 212/420] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:55.226172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:55.227310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:55.227806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:55.228276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:55.229270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:55.229805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:55.231375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:55.242137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 209/421] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:55.249665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2763cf-a900-4d7c-80f1-e027ff60eee8 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:55.254394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:55.305822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:55.376626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403ee27-7114-4466-85ed-62295806cf7e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:55.416911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2763cf-a900-4d7c-80f1-e027ff60eee8 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:55.421573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:55.422395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:55.422685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:55.422964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:55.447387 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:55.463397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:55.464267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:55.465514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 210/422] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:55.470257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:55.492281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:55.493010 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:55.493300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:55.493581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:55.567260 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:55.579239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:55.582191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:55.694627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 213/423] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:55.701021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:55.712583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:55.717698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb1584-876c-4536-96ae-ea8e8a31a37e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:55.771934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:55.772651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:55.772937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:55.773216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:55.774026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:55.774334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:55.775796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:55.794321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 211/424] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:55.800110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:55.803062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:55.826406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad549189-7889-4aaf-83ad-5f517a3ba8be None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:55.885866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:55.886732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:55.887011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:55.887281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:55.890805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:55.892297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:55.895277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 214/425] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:55 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:55.901344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:55.988536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad501f1e-92d9-4b0d-9d5d-bf89c0dbdbbd None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.052294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60941749-68f7-46d2-866f-2bd96cf535a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.052979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60941749-68f7-46d2-866f-2bd96cf535a1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.053264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60941749-68f7-46d2-866f-2bd96cf535a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.053540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60941749-68f7-46d2-866f-2bd96cf535a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.082042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39fc69-abc8-48fa-bbf0-dc9082d2ed47 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.115941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:56.152127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a5514-0313-40f8-99bd-1795496d6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.153196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a5514-0313-40f8-99bd-1795496d6962 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.153578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a5514-0313-40f8-99bd-1795496d6962 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.154240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a5514-0313-40f8-99bd-1795496d6962 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.155703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a5514-0313-40f8-99bd-1795496d6962 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:56.157886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a5514-0313-40f8-99bd-1795496d6962 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:56.159612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 215/426] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:55 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:56.161301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee9f0-4e01-4e22-8226-e39c808fd53d None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.161410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee9f0-4e01-4e22-8226-e39c808fd53d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.161728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee9f0-4e01-4e22-8226-e39c808fd53d None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.167418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 216/427] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:56.167418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ee9f0-4e01-4e22-8226-e39c808fd53d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:56.174303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60941749-68f7-46d2-866f-2bd96cf535a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:56.176132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60941749-68f7-46d2-866f-2bd96cf535a1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:56.198178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:56.288232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ee9f0-4e01-4e22-8226-e39c808fd53d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.315931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 212/428] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 17:29:56.317322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154babcc-b9f4-4211-92e4-3e96892517b2 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.317602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154babcc-b9f4-4211-92e4-3e96892517b2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.317873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154babcc-b9f4-4211-92e4-3e96892517b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.326164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afa9c6-96c7-491d-bcd9-747bd939f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.326820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afa9c6-96c7-491d-bcd9-747bd939f32c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.327104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afa9c6-96c7-491d-bcd9-747bd939f32c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.327377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afa9c6-96c7-491d-bcd9-747bd939f32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.328200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afa9c6-96c7-491d-bcd9-747bd939f32c None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:56.328543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afa9c6-96c7-491d-bcd9-747bd939f32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:56.330468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afa9c6-96c7-491d-bcd9-747bd939f32c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:56.334461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-154babcc-b9f4-4211-92e4-3e96892517b2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:56.335685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-154babcc-b9f4-4211-92e4-3e96892517b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:56.337416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 217/429] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:56.338474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331081-6ac0-4a54-a37b-b088243d1c9a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.338756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331081-6ac0-4a54-a37b-b088243d1c9a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.339031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331081-6ac0-4a54-a37b-b088243d1c9a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.344272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 218/430] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:56.344272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0331081-6ac0-4a54-a37b-b088243d1c9a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:56.420672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:56.441229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 213/431] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:56.442578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce723-1f7f-4b16-8b1c-ecad28a0f0bc None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.442872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce723-1f7f-4b16-8b1c-ecad28a0f0bc None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.443153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce723-1f7f-4b16-8b1c-ecad28a0f0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.470847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5b0ce723-1f7f-4b16-8b1c-ecad28a0f0bc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:56.472062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5b0ce723-1f7f-4b16-8b1c-ecad28a0f0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:56.487580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:56.500799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0331081-6ac0-4a54-a37b-b088243d1c9a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.552508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.553316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.556459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.556459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] PATH_INFO: `/v3/projects/c12a43fae56142df9ddb88b6d5c798fb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.556647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 214/432] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:56.556647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ce723-1f7f-4b16-8b1c-ecad28a0f0bc None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:56.560157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a43fae56142df9ddb88b6d5c798fb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:56.561665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:56.564354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 219/433] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/projects/c12a43fae56142df9ddb88b6d5c798fb => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:56.569576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:56.691062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ce723-1f7f-4b16-8b1c-ecad28a0f0bc None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.706956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f2cf5-2cc6-4157-a072-23cc66769930 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.721767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:56.741262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.742073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.742401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.742684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.747106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.747960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.748248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.748558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.749741 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:56.751185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:56.753696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 220/434] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:56.754384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:56.758253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:56.766814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:56.767751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:29:56.768572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 215/435] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:56.774441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:56.793675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:56.878973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872867ed-8224-4bde-a10c-d0a3922d7458 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.896627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6ed95-ba24-4825-82b3-b8211ac2c78a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:56.914083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.915418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.915704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.915981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.926652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:56.932697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:56.933345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:56.933627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:56.933903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:56.934732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=c12a43fae56142df9ddb88b6d5c798fb) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:56.935042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:56.935352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:56.936219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:29:56.937350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:56.937966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:56.938033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 221/436] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:29:56.938033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:56.942758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 216/437] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=c12a43fae56142df9ddb88b6d5c798fb&role.id=4182f79f5cda459497c049f3f07a1713 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:56.943478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.016256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.047342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.061555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44879566-cc37-4067-b10b-bad3b2fcbe9b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.099920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.100764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.104474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.104888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.105764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] RBAC: Adding query filter params (name=member) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:57.106167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.107662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.111067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9561ad-953f-40bb-95be-d2d73f838a22 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.121871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 222/438] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:57.123127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.161202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.162441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.163327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.163327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.165232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:57.165649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.167173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.182654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 217/439] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:29:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:57.189455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.258928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd3424-c009-4721-a635-b91ad53c3398 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.308802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de344a6-acf6-449c-8f67-2fef44c781b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.309489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de344a6-acf6-449c-8f67-2fef44c781b1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.309776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de344a6-acf6-449c-8f67-2fef44c781b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.310055 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de344a6-acf6-449c-8f67-2fef44c781b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.321266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1a4f4-6a10-403a-bdf2-206952e64906 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.334183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.334562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.368591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71953f20-54c5-4870-80eb-5c0ed3172660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.369254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953f20-54c5-4870-80eb-5c0ed3172660 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.369542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953f20-54c5-4870-80eb-5c0ed3172660 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.369820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953f20-54c5-4870-80eb-5c0ed3172660 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.403130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de344a6-acf6-449c-8f67-2fef44c781b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.405710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de344a6-acf6-449c-8f67-2fef44c781b1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.461227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71953f20-54c5-4870-80eb-5c0ed3172660 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.462599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71953f20-54c5-4870-80eb-5c0ed3172660 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.512715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 223/440] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:57.513961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db8f39-1f1b-4962-9247-d14d23456cad None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.514255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db8f39-1f1b-4962-9247-d14d23456cad None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.514536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db8f39-1f1b-4962-9247-d14d23456cad None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.516566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 224/441] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:29:57.517883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db8f39-1f1b-4962-9247-d14d23456cad None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.560921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 218/442] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:57.560921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a159141-e368-4e12-a276-ca2f6c6f864b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.560921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a159141-e368-4e12-a276-ca2f6c6f864b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.560921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a159141-e368-4e12-a276-ca2f6c6f864b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.573903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.577587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-7a159141-e368-4e12-a276-ca2f6c6f864b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:29:57.578771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7a159141-e368-4e12-a276-ca2f6c6f864b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:29:57.581265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.624533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db8f39-1f1b-4962-9247-d14d23456cad None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.643845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 219/443] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:29:57.644682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a159141-e368-4e12-a276-ca2f6c6f864b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.656758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.657882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.658171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.658454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.659632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.661105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.662842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 225/444] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:29:57.667316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.756864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a159141-e368-4e12-a276-ca2f6c6f864b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.764665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fba30c-9d32-43d8-bcfc-45a5fef6d1d7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.781076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.788459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.789892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.790176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.790455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] PATH_INFO: `/v3/projects/4f8537768c1b489d865a39e704eb9b79` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.791724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] RBAC: Authorizing `identity:get_project(project_id=4f8537768c1b489d865a39e704eb9b79)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.793285 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.795157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 220/445] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/projects/4f8537768c1b489d865a39e704eb9b79 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:57.796012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.798882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.798994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.799359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.799755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.804961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:57.810248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:29:57.817739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:29:57.818659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:29:57.819479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 226/446] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:29:57.820153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.897434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b55b6-3a39-457e-b4ae-10630136a7ef None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.920811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c34e48-ae5e-44f6-8952-710f2e4d51af None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:57.930149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.931011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.931301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.931583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.932872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.934305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.935938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 221/447] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:57.936712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:57.954308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:57.955157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:57.955448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:57.955726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] PATH_INFO: `/v3/groups/794f0ffb3816401baac5bbbd399580e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:57.956935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] RBAC: Authorizing `identity:get_group(group_id=794f0ffb3816401baac5bbbd399580e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:57.958369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:57.959971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 227/448] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/groups/794f0ffb3816401baac5bbbd399580e8 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:57.960739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:58.005565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:58.010826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:58.034258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3932f-6a83-45e7-b925-34df32162221 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:58.057231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca5d3d-02f2-4ec8-9cf4-e0734dcc2e98 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:58.066546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:58.067207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:58.067508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:58.067796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:58.068585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:58.069264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:58.071784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:58.080199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 222/449] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:58.080866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:58.091442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:58.092090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:58.092378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:58.092696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:58.093523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=295c91f6f2554a0ea6f954d898f9725c, scope.project.id=4f8537768c1b489d865a39e704eb9b79) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:58.093840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:58.095278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:58.124461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 228/450] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:57 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=295c91f6f2554a0ea6f954d898f9725c => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:58.124461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:58.187773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29510f71-c04a-4d50-92ec-3505dd4ad700 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:58.221719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f4c69-e2f4-4542-a033-1e8a90fce73d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:58.224008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:58.224726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:58.225016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:58.225308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:58.226158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] RBAC: Adding query filter params (group.id=794f0ffb3816401baac5bbbd399580e8, role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=4f8537768c1b489d865a39e704eb9b79) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:58.226474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:58.227962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:58.231692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:58.234753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:58.234993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 223/451] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:58 2022] GET /identity/v3/role_assignments?group.id=794f0ffb3816401baac5bbbd399580e8&scope.project.id=4f8537768c1b489d865a39e704eb9b79&role.id=4182f79f5cda459497c049f3f07a1713 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:58.262106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:58.262701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:58.262983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:58.263258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:58.358666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:58.360190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:58.470636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 229/452] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:29:58.476349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:58.578199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1543d60-e3da-4a93-a526-9a6247a88638 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:58.612269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:58.613174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:58.613460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:58.613738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] PATH_INFO: `/v3/projects/3ddf29534d9849ed9221c0b30b65b542` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:58.614921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] RBAC: Authorizing `identity:get_project(project_id=3ddf29534d9849ed9221c0b30b65b542)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:58.616383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:58.618169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 224/453] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:29:58 2022] GET /identity/v3/projects/3ddf29534d9849ed9221c0b30b65b542 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:58.623703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:58.635796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:58.727516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243efe14-679b-4631-9c46-baad989b3192 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:58.760375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53e808-7278-4498-8d34-7e85c598b549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:58.761254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53e808-7278-4498-8d34-7e85c598b549 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:58.761539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53e808-7278-4498-8d34-7e85c598b549 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:58.761817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53e808-7278-4498-8d34-7e85c598b549 None admin] PATH_INFO: `/v3/groups/9b763f0ed8834c66bad11d23e7b2fa4b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:58.762976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53e808-7278-4498-8d34-7e85c598b549 None admin] RBAC: Authorizing `identity:get_group(group_id=9b763f0ed8834c66bad11d23e7b2fa4b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:58.764450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53e808-7278-4498-8d34-7e85c598b549 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:58.766048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 230/454] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Jul 20 17:29:58 2022] GET /identity/v3/groups/9b763f0ed8834c66bad11d23e7b2fa4b => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:29:58.771901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:29:58.837786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:29:58.869956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b004d2-ab4a-49fe-b90e-0efa9113d65f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:29:58.901729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413efd-04fe-4eee-814e-a3f1d722818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:29:58.902370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413efd-04fe-4eee-814e-a3f1d722818d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:29:58.902660 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413efd-04fe-4eee-814e-a3f1d722818d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:29:58.902941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413efd-04fe-4eee-814e-a3f1d722818d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:29:58.903852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413efd-04fe-4eee-814e-a3f1d722818d None admin] RBAC: Adding query filter params (group.id=9b763f0ed8834c66bad11d23e7b2fa4b, role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=3ddf29534d9849ed9221c0b30b65b542) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:29:58.904175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413efd-04fe-4eee-814e-a3f1d722818d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:29:58.905641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413efd-04fe-4eee-814e-a3f1d722818d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:29:58.911547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 225/455] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Jul 20 17:29:58 2022] GET /identity/v3/role_assignments?group.id=9b763f0ed8834c66bad11d23e7b2fa4b&scope.project.id=3ddf29534d9849ed9221c0b30b65b542&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:01.543349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad28c5c-0794-4a20-8be7-e70b7bff8245 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:01.543349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad28c5c-0794-4a20-8be7-e70b7bff8245 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:01.543349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad28c5c-0794-4a20-8be7-e70b7bff8245 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:01.544382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 231/456] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:01.553542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864950c-b4b8-46fe-b89a-fcc2fe87ca11 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:01.553542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864950c-b4b8-46fe-b89a-fcc2fe87ca11 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:01.553542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864950c-b4b8-46fe-b89a-fcc2fe87ca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:01.574265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c864950c-b4b8-46fe-b89a-fcc2fe87ca11 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:01.575814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c864950c-b4b8-46fe-b89a-fcc2fe87ca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:01.665501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 226/457] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:30:01.694920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b05258-8880-4d8b-a94a-76bd19ad8615 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:01.695099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8790b-6578-4891-9d03-e191a973bca8 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:01.695176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b05258-8880-4d8b-a94a-76bd19ad8615 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:01.695269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8790b-6578-4891-9d03-e191a973bca8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:01.695999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b05258-8880-4d8b-a94a-76bd19ad8615 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:01.696112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8790b-6578-4891-9d03-e191a973bca8 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:01.698516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 227/459] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:01.698516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8790b-6578-4891-9d03-e191a973bca8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:01.698702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 232/459] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:01.703819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fab30-52eb-489a-ae18-5b13e92b0c2e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:01.704570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fab30-52eb-489a-ae18-5b13e92b0c2e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:01.705057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fab30-52eb-489a-ae18-5b13e92b0c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:01.728367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-6f7fab30-52eb-489a-ae18-5b13e92b0c2e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:01.730225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6f7fab30-52eb-489a-ae18-5b13e92b0c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:01.795024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:01.857312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 233/460] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:01.864398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baba383-63e5-4ccb-bd02-3c4d803c59e1 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:01.864398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baba383-63e5-4ccb-bd02-3c4d803c59e1 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:01.864398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baba383-63e5-4ccb-bd02-3c4d803c59e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:01.869900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8790b-6578-4891-9d03-e191a973bca8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:01.891811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1baba383-63e5-4ccb-bd02-3c4d803c59e1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:01.893916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1baba383-63e5-4ccb-bd02-3c4d803c59e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:01.898342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:01.937544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:01.938679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:01.939068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:01.939445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:01.956602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:01.959343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:01.991684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:01.993339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:01.994198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:01.994198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:01.994198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:01.994198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:01.999793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 228/461] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:01.999793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.010919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 234/462] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:02.012831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baba383-63e5-4ccb-bd02-3c4d803c59e1 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.126924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f94d-b1b1-4728-9059-04e80965c8aa None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.142203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:02.150762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baba383-63e5-4ccb-bd02-3c4d803c59e1 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.167881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.169184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.170042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.170796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.184872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.187338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:02.190876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.191555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.191845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.192123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.192883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:02.193194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.194566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:02.207223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:02.217442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:02.218332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:02.219179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:02.219179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:02.219179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:02.219179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:02.219179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 229/463] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:02.219179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.230903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 235/464] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:02.232710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.339667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cfd0e-1363-4760-a7e8-9fd9325f8582 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.361056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a388f4-1185-42dd-b98a-39ccbda0dddc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.375898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.376728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.377189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.377472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.388091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.389600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:02.396613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35451c-4fa4-48fe-bb30-faf4ff29b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.397285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35451c-4fa4-48fe-bb30-faf4ff29b884 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.397574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35451c-4fa4-48fe-bb30-faf4ff29b884 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.397854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35451c-4fa4-48fe-bb30-faf4ff29b884 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.398614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35451c-4fa4-48fe-bb30-faf4ff29b884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:02.398922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35451c-4fa4-48fe-bb30-faf4ff29b884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.400340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35451c-4fa4-48fe-bb30-faf4ff29b884 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:02.408724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:02.409737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:02.410643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:02.410643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:02.410643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:02.410643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:02.410643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 230/465] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:02.411472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.415512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 236/466] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:02.417441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe94aa-8cd6-49e6-b665-129ebdcdcd38 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.417955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe94aa-8cd6-49e6-b665-129ebdcdcd38 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.419331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe94aa-8cd6-49e6-b665-129ebdcdcd38 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.420821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 237/467] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:02.421706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe94aa-8cd6-49e6-b665-129ebdcdcd38 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.435809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:02.491697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:02.534468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a466a47-c741-4665-b9bf-1f0c691c7949 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.552773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe94aa-8cd6-49e6-b665-129ebdcdcd38 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.571595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16c62e-6503-4131-81e8-cd22be652074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.572267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16c62e-6503-4131-81e8-cd22be652074 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.572592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16c62e-6503-4131-81e8-cd22be652074 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.572867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16c62e-6503-4131-81e8-cd22be652074 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.573570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16c62e-6503-4131-81e8-cd22be652074 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.575023 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16c62e-6503-4131-81e8-cd22be652074 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:02.590329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.591093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.591455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.591796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.592622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:02.593000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.594491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:02.606298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 238/468] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:02.607144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.672940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 231/469] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:30:02 2022] POST /identity/v3/users => generated 246 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:02.674550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f0347-0c38-454c-8d76-2ad48aa84ad8 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.674978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f0347-0c38-454c-8d76-2ad48aa84ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.675334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f0347-0c38-454c-8d76-2ad48aa84ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.705823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3b2f0347-0c38-454c-8d76-2ad48aa84ad8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:02.707155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3b2f0347-0c38-454c-8d76-2ad48aa84ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:02.720890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155db93-0d92-4b0c-a867-c5d4c1803205 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.741368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:02.763388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec17b6-8186-4852-b471-a6d18762c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.764501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec17b6-8186-4852-b471-a6d18762c638 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.765029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec17b6-8186-4852-b471-a6d18762c638 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.765524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec17b6-8186-4852-b471-a6d18762c638 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.766444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec17b6-8186-4852-b471-a6d18762c638 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.768116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec17b6-8186-4852-b471-a6d18762c638 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:02.797411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 232/470] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:02.798154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f0347-0c38-454c-8d76-2ad48aa84ad8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.806337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:02.859220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 239/471] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:30:02 2022] POST /identity/v3/users => generated 248 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:02.860900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d6eb-0461-490a-9d94-9a1976cc9ee4 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.862615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d6eb-0461-490a-9d94-9a1976cc9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.862615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d6eb-0461-490a-9d94-9a1976cc9ee4 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.865169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 240/472] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:02.865169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d6eb-0461-490a-9d94-9a1976cc9ee4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:02.935143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f0347-0c38-454c-8d76-2ad48aa84ad8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:02.981526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:02.982522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:02.984271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:02.984271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:02.984271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:02.987087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:03.010418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d6eb-0461-490a-9d94-9a1976cc9ee4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:03.047884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8d921-4ecc-4766-a115-99537488a608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:03.048720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d921-4ecc-4766-a115-99537488a608 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:03.049000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d921-4ecc-4766-a115-99537488a608 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:03.049272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8d921-4ecc-4766-a115-99537488a608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:03.060673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8d921-4ecc-4766-a115-99537488a608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:03.062140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8d921-4ecc-4766-a115-99537488a608 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-77a8d921-4ecc-4766-a115-99537488a608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:03.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:03.083354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:03.084207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:03.084207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:03.084207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:03.084207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:03.084207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 241/473] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:03.084923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a304dd-ab7a-4e12-90d7-74c7011f141d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:03.085021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a304dd-ab7a-4e12-90d7-74c7011f141d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:03.088458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a304dd-ab7a-4e12-90d7-74c7011f141d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:03.104160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b3a304dd-ab7a-4e12-90d7-74c7011f141d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:03.105609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b3a304dd-ab7a-4e12-90d7-74c7011f141d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:03.128022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:03.143218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 233/474] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:30:02 2022] POST /identity/v3/users => generated 248 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:03.143922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:03.231698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:03.251271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 242/475] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:03.291908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149714cf-0d4f-41bf-ab3f-c43a00a3b502 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:03.297467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a304dd-ab7a-4e12-90d7-74c7011f141d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:03.344591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:03.345263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:03.345599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:03.345886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:03.346684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:03.346995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:03.348459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:03.362272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 234/476] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:03.368340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:03.423758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a304dd-ab7a-4e12-90d7-74c7011f141d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:03.465089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:03.465882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:03.466168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:03.466562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:03.467320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:03.468752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:03.493341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2564e-5415-4057-b018-de2288eb0e67 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:03.521659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:03.530373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.sql.core [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 17:30:03.530373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 17:30:03.530373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [parameters: {'id': 'a16736caeb444d1fb22aafcefa5cef82', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 17:30:03.530373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80583) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 17:30:03.548965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:03.549905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:03.550334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:03.551691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:03.553060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:03.556890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application conn.query(q) Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result.read() Jul 20 17:30:03.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 17:30:03.567023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application next(self.gen) Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application yield resource Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application next(self.gen) Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise self.value Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application session.commit() Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 17:30:03.568160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.session.flush() Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application compat.raise_( Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 17:30:03.569347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 17:30:03.570455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application conn.query(q) Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result.read() Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 17:30:03.571563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [parameters: {'id': 'a16736caeb444d1fb22aafcefa5cef82', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:03.572933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 17:30:03.574713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:03.576754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 243/477] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jul 20 17:30:03 2022] POST /identity/v3/roles => generated 144 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:03.576754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:03.604446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:03.691805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 235/478] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:30:03 2022] POST /identity/v3/users => generated 249 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:30:03.693921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984e466-ce0d-4884-8027-95052dda2e07 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:03.751436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11a2ba-4270-402b-bc6a-0927bbec8fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:03.752191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11a2ba-4270-402b-bc6a-0927bbec8fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:03.752502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11a2ba-4270-402b-bc6a-0927bbec8fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:03.752790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11a2ba-4270-402b-bc6a-0927bbec8fc2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:03.773036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11a2ba-4270-402b-bc6a-0927bbec8fc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-be11a2ba-4270-402b-bc6a-0927bbec8fc2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:03.782582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:03.783579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:03.784385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 244/479] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:03.789682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:03.914091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3623f64-50e3-4143-a992-640f79ccc325 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:03.948388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7cca7-090e-4319-98f4-f9fda5d02d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:03.949091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7cca7-090e-4319-98f4-f9fda5d02d79 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:03.949378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7cca7-090e-4319-98f4-f9fda5d02d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:03.949657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7cca7-090e-4319-98f4-f9fda5d02d79 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:03.950476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7cca7-090e-4319-98f4-f9fda5d02d79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:03.950786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7cca7-090e-4319-98f4-f9fda5d02d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:03.952282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c7cca7-090e-4319-98f4-f9fda5d02d79 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:03.961009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 236/480] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:05.146020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e4d1-7431-4499-9690-b92b57a45b25 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.146848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e4d1-7431-4499-9690-b92b57a45b25 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.147300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e4d1-7431-4499-9690-b92b57a45b25 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.149713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 245/481] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:05.154751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c35bc5-7dad-4ed6-a5c6-ad204d4d56a7 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.155476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c35bc5-7dad-4ed6-a5c6-ad204d4d56a7 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.156064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c35bc5-7dad-4ed6-a5c6-ad204d4d56a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.181456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b8c35bc5-7dad-4ed6-a5c6-ad204d4d56a7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:05.182895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b8c35bc5-7dad-4ed6-a5c6-ad204d4d56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:05.237540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:05.273836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e56da-4672-408a-9bba-9a4d4c7cb823 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.274460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e56da-4672-408a-9bba-9a4d4c7cb823 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.274945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e56da-4672-408a-9bba-9a4d4c7cb823 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.276802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 246/482] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:30:05.284553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 237/483] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:05.298312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50886561-c521-49bb-bb57-a2c6dcaef2e6 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.298851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50886561-c521-49bb-bb57-a2c6dcaef2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.299293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50886561-c521-49bb-bb57-a2c6dcaef2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.316966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c35bc5-7dad-4ed6-a5c6-ad204d4d56a7 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:05.324754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-50886561-c521-49bb-bb57-a2c6dcaef2e6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:05.325960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-50886561-c521-49bb-bb57-a2c6dcaef2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:05.419948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c35bc5-7dad-4ed6-a5c6-ad204d4d56a7 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:05.457115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 247/484] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:05.469146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:05.470242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.470755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.471233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.494096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:05.507535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50886561-c521-49bb-bb57-a2c6dcaef2e6 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:05.518941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:05.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:05.520756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:05.527026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 238/485] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:05.531807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:05.533223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:05.592818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:05.702583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50886561-c521-49bb-bb57-a2c6dcaef2e6 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:05.706180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e640b-ad24-4b1a-bf2b-ed7ffa4a1cab None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:05.738228 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:05.739023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.739305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.739577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.741814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b35587-70f4-4a07-8b4e-c40635190e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:05.743192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b35587-70f4-4a07-8b4e-c40635190e81 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.743653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b35587-70f4-4a07-8b4e-c40635190e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.744103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b35587-70f4-4a07-8b4e-c40635190e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.745084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b35587-70f4-4a07-8b4e-c40635190e81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:05.745687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b35587-70f4-4a07-8b4e-c40635190e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:05.747302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b35587-70f4-4a07-8b4e-c40635190e81 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:05.750555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:05.756137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 239/486] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:05.764460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267915a8-6f83-4fd8-a5c9-9e5933b4fa2c None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:05.764612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:05.765465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:05.766361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 248/487] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:05.767016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267915a8-6f83-4fd8-a5c9-9e5933b4fa2c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.767016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267915a8-6f83-4fd8-a5c9-9e5933b4fa2c None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.767016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 240/488] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:05.770119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:05.770323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c97fbb-4fa6-44d3-9f46-293acd9e9fae None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.770605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c97fbb-4fa6-44d3-9f46-293acd9e9fae None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.770880 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c97fbb-4fa6-44d3-9f46-293acd9e9fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.790324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-77c97fbb-4fa6-44d3-9f46-293acd9e9fae None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:05.791949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-77c97fbb-4fa6-44d3-9f46-293acd9e9fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:05.843549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:05.862571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 241/489] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:05.863715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c97fbb-4fa6-44d3-9f46-293acd9e9fae None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:05.878438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3034964-e28e-4b7d-9335-06b662d443fa None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:05.887342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:05.917277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:05.918399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:05.918825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:05.919341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:05.931035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:05.932955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:05.951617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:05.952561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:05.953459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:05.953459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:05.953459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:05.953459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:05.955843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 249/490] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:05.956864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:05.978093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c97fbb-4fa6-44d3-9f46-293acd9e9fae None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.015124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.015798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.016107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.016361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.017393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:06.017571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.018956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.032779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 242/491] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:06.033816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.073663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0bfd9-1741-438a-8719-2cdb551bcc45 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.099827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.113304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.114875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.115360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.115847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.129717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:06.135830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:06.140777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:06.143258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:06.144064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:06.145451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 250/492] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:06.146473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.161259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b2ead-2645-4b6f-8248-527cb94a1e0a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.198990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.199641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.199935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.200214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.201250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:06.201532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.202922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.220701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 243/493] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:06.221544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.259486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c170f4-03bf-4ead-8b2a-94a534f8aa4a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.303669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.305164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.305465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.305820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.320462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.322590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.328153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6928-4deb-4a5f-9806-2d006d7cd8c1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:06.345916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:06.346806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:06.347648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:06.347648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:06.347648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:06.347648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:06.351297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 251/494] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:06.352463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.400064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.400831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.401128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.401400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.402139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:06.402445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.402947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.403846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.417435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 244/495] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:06.417435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.422263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.483111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efad675-cbd3-4e6a-b276-2f965984df06 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.518829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.520452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.520452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.520452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.528727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.530182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:06.547248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:06.548224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:06.549278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 252/496] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:06.549867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.558724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e88716-54b3-4550-a922-355683ee0adb None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.592009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce15505-d44d-4d46-9824-a34b27e9ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.592697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15505-d44d-4d46-9824-a34b27e9ab64 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.592983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15505-d44d-4d46-9824-a34b27e9ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.593262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15505-d44d-4d46-9824-a34b27e9ab64 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.593985 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce15505-d44d-4d46-9824-a34b27e9ab64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:06.594293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce15505-d44d-4d46-9824-a34b27e9ab64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.595667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce15505-d44d-4d46-9824-a34b27e9ab64 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.607200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 245/497] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:06.608766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76a4cb-b05d-4d62-bf20-b22778a67732 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.609120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76a4cb-b05d-4d62-bf20-b22778a67732 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.609386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76a4cb-b05d-4d62-bf20-b22778a67732 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.612477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 246/498] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:06.612477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89799e-b84e-4ddf-b3bb-b91884aae962 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.612477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89799e-b84e-4ddf-b3bb-b91884aae962 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.612754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89799e-b84e-4ddf-b3bb-b91884aae962 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.613914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 247/499] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:06.618465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89799e-b84e-4ddf-b3bb-b91884aae962 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.644410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.651449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096a3c4-f808-471e-ab5a-8ec0f80317bc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.678834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.686401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.687197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.687478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.687748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.698198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.699686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:06.720950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:06.721862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:06.722702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:06.722702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:06.722702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:06.722702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:06.722911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89799e-b84e-4ddf-b3bb-b91884aae962 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.722980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 253/500] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:06.723339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.756410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a4d8e-b4c0-4902-b996-54284d7dfa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.757049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a4d8e-b4c0-4902-b996-54284d7dfa64 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.757332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a4d8e-b4c0-4902-b996-54284d7dfa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.758134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a4d8e-b4c0-4902-b996-54284d7dfa64 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.758954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a4d8e-b4c0-4902-b996-54284d7dfa64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:06.759264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a4d8e-b4c0-4902-b996-54284d7dfa64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.760758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a4d8e-b4c0-4902-b996-54284d7dfa64 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.797058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 248/501] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:06.798361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b8db-07c0-46d5-9737-0067cbaf3941 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.798653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b8db-07c0-46d5-9737-0067cbaf3941 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.798932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574b8db-07c0-46d5-9737-0067cbaf3941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.816089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-8574b8db-07c0-46d5-9737-0067cbaf3941 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:06.817409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8574b8db-07c0-46d5-9737-0067cbaf3941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:06.830210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c451dd-1f95-4e62-9c7b-c1395a6e4535 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:06.864911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:06.865686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.865966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.866240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.873031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:06.874472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:06.884634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 249/502] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:06.885997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195deb1-0e28-4d17-a93e-dfd8f661944b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:06.886288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195deb1-0e28-4d17-a93e-dfd8f661944b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:06.886570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195deb1-0e28-4d17-a93e-dfd8f661944b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:06.889476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:06.890527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:06.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 254/503] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:06.892036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.907335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.908233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-2195deb1-0e28-4d17-a93e-dfd8f661944b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:06.909505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2195deb1-0e28-4d17-a93e-dfd8f661944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:06.954219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:06.973212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 250/504] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:06.974067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195deb1-0e28-4d17-a93e-dfd8f661944b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:06.998869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c7e21-6849-4ae7-b2f0-fc3ba962a998 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.039615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.040527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.041005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.041411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.042315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:07.043058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.044621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.056840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 255/505] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:07.057829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.089183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195deb1-0e28-4d17-a93e-dfd8f661944b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.128487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.129068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.129734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.130014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.130293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.163916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45077556-3e23-48c5-a1d8-5caa735a7c97 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.185674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.205119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.207034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.207879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.208592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.208882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.209158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.210162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:07.210475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.212530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.248034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 256/506] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:07.248953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.304321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 251/507] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:07.305662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.345576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8370b5-54c6-4956-b339-18fcd56343dd None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.369622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.379607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.380397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.380715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.383613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.384793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.391420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:07.399560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:07.400583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:07.401381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 257/508] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:07.402047 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0f7b4-19a8-4534-9561-6ff23a9b1431 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.402119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.451255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.452053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.452341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.452645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.462977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.464458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:07.484134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:07.485084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:07.485936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:07.485936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:07.485936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:07.485936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:07.485936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 252/509] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:07.486484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.532353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3dae52-bbb0-4a8b-b920-a734650733a0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.566879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.567601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.567880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.568157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.577648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.579117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:07.593364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:07.594279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:07.595189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 258/510] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:07.595762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.607776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af372f63-59ec-4535-a7ab-077f0dc5fec3 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.641394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.647720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.649040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.657651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.657651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.657651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.698439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ee5c2-44a8-4fd1-afb2-6a6ba67b8938 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.719352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.720993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.730896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.731682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.731967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.732234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.743135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.744716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:07.761750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:07.762591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:07.763455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:07.763455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:07.763455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:07.763455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:07.763455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 259/511] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:07.763715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.825014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 253/512] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:07.825773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.856079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.861974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422cdda-0d95-47a4-b97c-d2449d7a9843 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.867413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:07.894689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.895358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.895644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.895924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.896716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:07.897028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.898953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.912545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 260/513] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:07.912545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:07.937801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d75f9-635c-430d-a506-f804664b9ce7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:07.971950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:07.972643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:07.972924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:07.973204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:07.973923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:07.974226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:07.975581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:07.986890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 254/514] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:07.987651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.010482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d744d6-127e-448d-851f-3892ad271430 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.044338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.045048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.045337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.045617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.046454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:08.046766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.048237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.065329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 261/515] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:08.066109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.082781 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.087821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.094743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdf44-d634-46eb-8c50-bf49ee53a7e7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.128724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.129392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.129680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.129958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.132715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:08.132715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.132715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.143951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 255/516] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:08.144758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.164806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615f36f-242a-4870-a3b0-161f37c740cc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.204716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.205505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.205792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.206072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.219377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.221336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:08.240532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:08.241636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:08.247311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:08.247311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:08.247311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:08.247311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:08.247311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 262/517] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:08.247311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.248260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8b2a1-62bd-4656-802c-be7997fcac85 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.282379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.283161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.283439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.283710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.293665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.295106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.311953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:08.312987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:08.313846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:08.314715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:08.314715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:08.314715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:08.314715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:08.314715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 256/518] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:08.315003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.323819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.349340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312e4b1-27ab-4c3c-990a-d35f26d96d17 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.383264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.383928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.384209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.384507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.385200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.386706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.412124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3567b-29dd-4302-8c3f-442bfe05c71f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.446512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.447158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.447445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.447724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.489215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 263/519] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:30:08 2022] POST /identity/v3/users => generated 247 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:08.500882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.516823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.519245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.530221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.623746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d858ba1-43a0-4284-ab33-3a9c6059bed7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.637246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 257/520] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:08.638392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.654758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.669335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.670162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.670529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.670726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.682368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.688474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:08.701454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:08.702614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:08.703456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:08.703866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 264/521] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:08.705226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.765885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3aa3b-e567-4994-9c53-e8d48f4e2a3f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.777681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.804329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.805291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.805941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.807136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.807881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:08.808194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.809644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.817037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b094dc0-bb31-4e4d-8f50-ce9e43d73927 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.825590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 258/522] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:08.825590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.862562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.863231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.863515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.863914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.864687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:08.864995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.866360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.886378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 265/523] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:08.886378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:08.932341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:08.937702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8bc6c-dab8-402c-987c-4b4fbe2dfc4d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:08.972995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:08.974021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:08.974367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:08.974708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:08.987583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:08.989854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:08.991093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf279be-9d69-4e42-9c5e-12b7ec435de3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:09.009725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:09.010823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:09.012001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:09.012001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:09.012001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:09.012001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:09.012788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 259/524] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:09.017399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.033833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:09.039949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.040950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.041492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.041775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.042733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:09.043427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.045342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.067743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 266/525] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:09.073615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.126062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628be55-1cbd-494d-9898-63731296b5bb None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.159888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.160798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.161154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.161499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.169733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.171256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:09.187593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:09.188491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:09.189459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:09.189871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb42bf-dc30-4178-82af-5772eda6298e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.190263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 260/526] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:09.190263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.194550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:09.229620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.230516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.230875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.231221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.239759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.241336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:09.257108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:09.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 267/527] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:09.263873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.278482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:09.301665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f8da1-58a0-4f93-b58f-9c7137f748b5 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.341915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.342575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.342856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.343133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.343848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:09.344154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.345549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.357164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 261/528] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:09.357756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.372458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64a9d-2798-462b-a0bb-f0e4cbdb478d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.407602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.408409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.408624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.408846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.409677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=de1a294634524a9d8ff48c7c50726228) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:09.410181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.411606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.418216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 268/529] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/role_assignments?user.id=de1a294634524a9d8ff48c7c50726228&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:09.418907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.459989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:09.466417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b31a69-8ad0-482e-969c-c52ee21daf6f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.505034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.506089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.506570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.507038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.508101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:09.508638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.510302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.525387 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:09.534055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 262/530] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:09.540655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.552145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411290b-badb-4fd7-b28f-b7570450a858 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.603830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.605176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.605567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.605942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.607320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:09.607737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.609901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.649420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 269/531] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:09.650660 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.678928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f12c4c-0c2c-43bc-bdaa-d8bffc4bbc45 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.716975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.718009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.718440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.718876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.726272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.727923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:09.743175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:09.744150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:09.745238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:09.751926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 263/532] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:09.752977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.776731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:09.793354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c7c5e-0a38-4427-a1a1-7a92f3676b85 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.825945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:09.828971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.829753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.830110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.830576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:09.831618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=b6f536917f434773a052aa074d5aa07d) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:09.831736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:09.833832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:09.844556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 270/533] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/role_assignments?user.id=b6f536917f434773a052aa074d5aa07d&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:09.844556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:09.917685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b0aed-3972-43dd-80d0-3cd4f4863847 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.991634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d8c0f-17e0-468d-a120-5ae3ac635beb None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:09.993288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:09.993962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:09.994249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:09.994530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.045766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:10.046855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:10.047370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:10.047843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.049852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:10.050752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:10.055089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:10.087988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 271/534] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:10.091812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:10.123474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:10.146151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:10.147627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:10.216899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:10.251600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 264/535] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:10.256704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:10.306052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9317c-27a0-44ff-8ef0-de93154bb906 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:10.359003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c198827-5a38-470a-9e14-c18d70938a38 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:10.393439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:10.394233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:10.394518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:10.394787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.398086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:10.398802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:10.399082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:10.399358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.400195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=9706ef808a224c83aca165c595b596ef) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:10.404581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:10.406088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:10.415010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:10.416476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:10.419957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 272/536] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:10 2022] GET /identity/v3/role_assignments?user.id=9706ef808a224c83aca165c595b596ef&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:10.448473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:10.449378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:10.450215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:10.450215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:10.450215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:10.450215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:10.450215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 265/537] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:10.453568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:10.580985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:10.617810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2eba3-df7f-4a1c-955b-75b4d3f7607e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:10.675604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695f1bb-7911-4dc9-b8b1-c08f9f8033ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:10.676277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f1bb-7911-4dc9-b8b1-c08f9f8033ee None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:10.676664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f1bb-7911-4dc9-b8b1-c08f9f8033ee None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:10.676953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f1bb-7911-4dc9-b8b1-c08f9f8033ee None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.677688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695f1bb-7911-4dc9-b8b1-c08f9f8033ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:10.678000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695f1bb-7911-4dc9-b8b1-c08f9f8033ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:10.679465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695f1bb-7911-4dc9-b8b1-c08f9f8033ee None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:10.692201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 273/538] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:10.697744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:10.710892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:10.826729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25fe3d-8475-4bb5-ac4e-eb761c9cd338 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:10.874114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:10.875374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:10.875663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:10.876256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.898810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204d35-116d-4585-97ef-c6a7d7c9f02f None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:10.899096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204d35-116d-4585-97ef-c6a7d7c9f02f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:10.899376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204d35-116d-4585-97ef-c6a7d7c9f02f None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.902386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 274/539] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:30:10.905262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:10.906295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05486-97bb-4fcf-bd37-83cc35680421 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:10.906590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05486-97bb-4fcf-bd37-83cc35680421 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:10.906757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:10.907169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05486-97bb-4fcf-bd37-83cc35680421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:10.934093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:10.935002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:10.935840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:10.936734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cbf05486-97bb-4fcf-bd37-83cc35680421 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:10.940459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cbf05486-97bb-4fcf-bd37-83cc35680421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:10.940647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 266/540] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:10.946505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:11.036797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:11.082385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a80c5-e7cf-4074-b261-3b89ac8d87ef None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:11.130411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:11.131087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:11.131380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:11.131661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:11.132530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:11.132858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:11.134329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:11.144537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 275/541] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:30:11.155950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 267/542] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:11.161933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05486-97bb-4fcf-bd37-83cc35680421 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:11.201822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:11.293116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:11.367136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1b0b8-e20a-49bf-8bca-106b23100fe4 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:11.394404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05486-97bb-4fcf-bd37-83cc35680421 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:11.401162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:11.417200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:11.418085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:11.418306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:11.418584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:11.434956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:11.448262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080bf1e-7b45-4157-8efe-ffefa7e1df87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:11.449124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080bf1e-7b45-4157-8efe-ffefa7e1df87 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:11.449546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080bf1e-7b45-4157-8efe-ffefa7e1df87 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:11.449990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080bf1e-7b45-4157-8efe-ffefa7e1df87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:11.450961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080bf1e-7b45-4157-8efe-ffefa7e1df87 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=d1d18a38f45e4501914e22689f145fcf) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:11.451425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080bf1e-7b45-4157-8efe-ffefa7e1df87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:11.453070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080bf1e-7b45-4157-8efe-ffefa7e1df87 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:11.453415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:11.454373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:11.455158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:11.455755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 268/543] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:11.461165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:11.468148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 276/544] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:11 2022] GET /identity/v3/role_assignments?user.id=d1d18a38f45e4501914e22689f145fcf&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:11.661504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6885c-2712-4ac0-9584-8e42181b83b7 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:11.661866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6885c-2712-4ac0-9584-8e42181b83b7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:11.662066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6885c-2712-4ac0-9584-8e42181b83b7 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:11.665347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 277/545] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:11.671050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e94d-6f1c-48d6-8eaa-00f8dde5427e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:11.671346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e94d-6f1c-48d6-8eaa-00f8dde5427e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:11.671633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e94d-6f1c-48d6-8eaa-00f8dde5427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:11.673943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f76c5-4872-4095-bc8d-bcd57ce39a11 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:11.701938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-eef0e94d-6f1c-48d6-8eaa-00f8dde5427e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:11.703274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-eef0e94d-6f1c-48d6-8eaa-00f8dde5427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:11.722994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:11.723678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:11.723962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:11.724241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:11.725097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:11.725408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:11.726858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:11.740684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 269/546] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:11.747246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:11.862210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 278/547] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:11.865527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:11.914464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0e94d-6f1c-48d6-8eaa-00f8dde5427e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:11.956037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c349e-1f09-4fd7-b0e0-255be841718d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:12.001589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:12.002399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.003089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.003376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.018039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:12.019601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:12.041508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:12.042484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:12.043341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:12.043341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:12.043341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:12.043341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:12.052592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 270/548] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:12.052592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:12.063204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:12.180643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff2808-0b3b-4c73-857b-755f77e502cf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:12.180643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:12.202796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0e94d-6f1c-48d6-8eaa-00f8dde5427e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:12.240028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:12.240623 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.240915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.241195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.242010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:12.244225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:12.248534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:12.280901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 271/549] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:12.285781 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:12.288207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4fc4-1bef-4dea-992f-b388b3242467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:12.289152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4fc4-1bef-4dea-992f-b388b3242467 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.289441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4fc4-1bef-4dea-992f-b388b3242467 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.289718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4fc4-1bef-4dea-992f-b388b3242467 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.317767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc4fc4-1bef-4dea-992f-b388b3242467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-98cc4fc4-1bef-4dea-992f-b388b3242467 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:12.334271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:12.335190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:12.336597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:12.343794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 279/550] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:12.346531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec166b-4932-4027-bce9-6e0088a87899 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.346829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec166b-4932-4027-bce9-6e0088a87899 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.347116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec166b-4932-4027-bce9-6e0088a87899 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.353362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 280/551] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:12.354089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec166b-4932-4027-bce9-6e0088a87899 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:12.511930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767ff9b-f9fa-4953-8c90-e010625e9453 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:12.562493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec166b-4932-4027-bce9-6e0088a87899 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:12.585865 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:12.609935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358bff3-d8ca-4b0b-b937-6c456e7106c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:12.611893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358bff3-d8ca-4b0b-b937-6c456e7106c5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.613186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358bff3-d8ca-4b0b-b937-6c456e7106c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.614924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358bff3-d8ca-4b0b-b937-6c456e7106c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.621746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358bff3-d8ca-4b0b-b937-6c456e7106c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:12.623297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358bff3-d8ca-4b0b-b937-6c456e7106c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:12.625632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358bff3-d8ca-4b0b-b937-6c456e7106c5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:12.643437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 281/552] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:12.643437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7205-16cb-4662-9f6f-a2f372ba1aa0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.643687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7205-16cb-4662-9f6f-a2f372ba1aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.648718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7205-16cb-4662-9f6f-a2f372ba1aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.653329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:12.654155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.654446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.654726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.671971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-fddc7205-16cb-4662-9f6f-a2f372ba1aa0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:12.673581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-fddc7205-16cb-4662-9f6f-a2f372ba1aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:12.682527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:12.688326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:12.709776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:12.711395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:12.712344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:12.716525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 272/553] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:12.717731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:12.764970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 282/554] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:12.766268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc7205-16cb-4662-9f6f-a2f372ba1aa0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:12.790379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:12.887507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b6208-2d42-498a-98a6-32dd98ed3d8b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:12.916778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc7205-16cb-4662-9f6f-a2f372ba1aa0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:12.929985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:12.937256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2ee01-a02e-42a6-a683-59ded805e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:12.938471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2ee01-a02e-42a6-a683-59ded805e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.941085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2ee01-a02e-42a6-a683-59ded805e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.941085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2ee01-a02e-42a6-a683-59ded805e2c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.952578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2ee01-a02e-42a6-a683-59ded805e2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:12.954122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2ee01-a02e-42a6-a683-59ded805e2c0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:12.956370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:12.957244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.957672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.958103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.959285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:12.960000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:12.962582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-39e2ee01-a02e-42a6-a683-59ded805e2c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:12.973604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:12.974670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:12.975681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:12.975681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:12.975681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:12.975681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:12.975681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 273/555] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:12.977261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069520-e2ea-4373-bea5-75b4add08206 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:12.977554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069520-e2ea-4373-bea5-75b4add08206 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:12.977831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069520-e2ea-4373-bea5-75b4add08206 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:12.983504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 274/556] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:12.984274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f069520-e2ea-4373-bea5-75b4add08206 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:12.986997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 283/557] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jul 20 17:30:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:12.986997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.097090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ab861-3176-46a8-9b40-1b9bbf74f3d9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.116199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.142601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf556ac-2733-4a58-9489-042039d866dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.143293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf556ac-2733-4a58-9489-042039d866dd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.143561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf556ac-2733-4a58-9489-042039d866dd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.143826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf556ac-2733-4a58-9489-042039d866dd None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.144699 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf556ac-2733-4a58-9489-042039d866dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:13.149178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf556ac-2733-4a58-9489-042039d866dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:13.149178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf556ac-2733-4a58-9489-042039d866dd None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:13.155000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f069520-e2ea-4373-bea5-75b4add08206 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.160795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 284/558] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:13.162054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190665c-6318-4029-9eab-611cdabe2194 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.162348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190665c-6318-4029-9eab-611cdabe2194 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.162630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190665c-6318-4029-9eab-611cdabe2194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.182283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7190665c-6318-4029-9eab-611cdabe2194 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:13.183723 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7190665c-6318-4029-9eab-611cdabe2194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:13.198403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.199598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.199893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.200176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.208743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.215326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:13.225294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:13.226831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 275/559] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:13.227701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.280456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 285/560] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:13.280456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190665c-6318-4029-9eab-611cdabe2194 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.362762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac960f7-74b4-428c-a1f1-e3554329b9d2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.383065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190665c-6318-4029-9eab-611cdabe2194 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.399619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.400613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.401128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.401605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.405319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.418543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.419187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.419473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.431914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.431914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:13.431914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:13.457941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:13.459013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 286/561] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:13.459972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.482788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.495439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:13.496940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:13.567901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46162-396d-46b8-86db-ef973963acaf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.617484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.618105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.618376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.618640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.619398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:13.619686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:13.621091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:13.630021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 287/562] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:13.630692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.681211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 276/563] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:13.682060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.721460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.731609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.751225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ae9a5-803b-44ac-86ab-5160f5114551 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.786559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.787722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.788002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.788280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.791185 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9142f0-e6d5-4cc1-b8bf-ca12c71ac92f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.799931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:13.807597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:13.808664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 288/564] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:13.809759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.832264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6603e3-1899-4dd2-bb6e-3f72a0d0b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.833169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6603e3-1899-4dd2-bb6e-3f72a0d0b8ff None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.833460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6603e3-1899-4dd2-bb6e-3f72a0d0b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.833741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6603e3-1899-4dd2-bb6e-3f72a0d0b8ff None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.834576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6603e3-1899-4dd2-bb6e-3f72a0d0b8ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:13.834893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6603e3-1899-4dd2-bb6e-3f72a0d0b8ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:13.836915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6603e3-1899-4dd2-bb6e-3f72a0d0b8ff None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:13.855637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 277/565] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:13.862082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f756c-6967-4950-af35-1a805e166fce None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.862384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f756c-6967-4950-af35-1a805e166fce None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.862845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f756c-6967-4950-af35-1a805e166fce None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.864479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 278/566] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:13.866694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f756c-6967-4950-af35-1a805e166fce None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:13.917836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bce03-c1d5-4622-ad75-672fe6c0b569 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.956156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:13.956986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:13.957266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:13.957548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:13.967701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.968778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:13.973987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:13.975340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f756c-6967-4950-af35-1a805e166fce None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:13.975454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:13.993129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:13.994026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:13.995134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:13.995134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:13.995134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:13.995134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:13.995134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 289/567] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:13.995572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.012980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.013777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.014199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.014428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.028385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.030753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:14.049108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:14.049999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:14.050881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:14.050881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:14.050881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:14.050881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:14.050881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 279/568] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:14.051172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.106519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211c0dd-2c95-4b24-9f0f-e74b5bc18ca8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.142519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76879afc-fcb6-4582-b6ee-1c2b657090f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.143584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879afc-fcb6-4582-b6ee-1c2b657090f1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.143950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879afc-fcb6-4582-b6ee-1c2b657090f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.144306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879afc-fcb6-4582-b6ee-1c2b657090f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.145197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76879afc-fcb6-4582-b6ee-1c2b657090f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:14.145579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76879afc-fcb6-4582-b6ee-1c2b657090f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.147758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76879afc-fcb6-4582-b6ee-1c2b657090f1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.158467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 290/569] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:14.158586 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817af64-b782-417c-9302-e8509fd0dbfd None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.164149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2373-7ca9-490f-b1f4-ae5d10e78fcd None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.164461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2373-7ca9-490f-b1f4-ae5d10e78fcd None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.164754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2373-7ca9-490f-b1f4-ae5d10e78fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.180365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-9d0c2373-7ca9-490f-b1f4-ae5d10e78fcd None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:14.181552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9d0c2373-7ca9-490f-b1f4-ae5d10e78fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:14.196180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.197119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.197405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.197673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.207771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.209364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.216956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:14.227897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:14.231757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:14.232771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:14.232771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:14.232771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:14.232771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:14.232771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 280/570] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:14.232771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.236893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.256464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 291/571] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:14.256464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2373-7ca9-490f-b1f4-ae5d10e78fcd None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.340103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d985eeb-220c-4c7a-92dc-347b2435770b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.368188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2373-7ca9-490f-b1f4-ae5d10e78fcd None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.373793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.374456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.374746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.375025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.375760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:14.376072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.377482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.388714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 281/572] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:14.389340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.404848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.405518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.405798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.406071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.406792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:14.407355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.408736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.420184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 292/573] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:14.420981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.451114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.465631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.491881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99311e46-5077-4d9f-9b94-041beddd5487 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.517875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4dade-108a-4166-b358-e134a9afd894 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.526195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.527090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.527379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.527655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.538744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.540232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.551333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.551992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.552280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.552592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.553317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:14.553627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.555498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:14.557111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:14.558095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:14.559021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:14.559021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:14.559021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:14.559021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:14.559021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 282/574] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:14.559600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.567343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 293/575] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:14.568203 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.661175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433ef55-eca3-48f8-ba5d-6dd6bb4aeef0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.666549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5670f6-01e1-47f6-965e-c5bbf037e025 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.668900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.671063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.697857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.698661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.698948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.699224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.700361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.701240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.701552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.701828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.709159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:14.717861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:14.718842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:14.719672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 283/576] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:14.720525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.724990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.727037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:14.741637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:14.742667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 294/577] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:14.743608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.830178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931309f-afd3-4c0a-8c33-a85b957d2bc2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.844115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ac0fd-8edb-4c87-b41f-947e147a1dc5 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.863010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.863801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.864085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.864358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.871321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.872790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.876836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:14.877492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:14.877777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:14.878057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:14.878775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:14.879083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:14.880456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:14.887219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:14.888070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:14.889140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 284/578] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:14.889984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.893882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 295/579] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:14.894690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:14.919880 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.923774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:14.988332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7129f9-fadc-4415-b429-6d76d7c617d2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:14.996459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dbb80-e25a-47a0-9862-5d2b634c48ce None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.021548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.022366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.022643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.022993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.030204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.032119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.036336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.037026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.037306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.037578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.038379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:15.038740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.040090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:15.046067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:15.047183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:15.048178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 285/580] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:15.053490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 296/581] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:15.053668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.053759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.137609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.147752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.153759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f154c-6215-4d9a-a3f6-26430f591306 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.157584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff662d8-4ef5-444a-9738-408901eea0e6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.187262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.187915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.188200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.188506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.189294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:15.189606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.189751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.190412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.190697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.190973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.191059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.191807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:15.192116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.193561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.203053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 297/582] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:15.203962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.212119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 286/583] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:15.213601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.302231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d656d-b34d-4503-9dbb-64b96df1599a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.313140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c31925-f724-46b3-9e32-e99257e3695e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.335133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.335950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.336230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.336538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.343754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.345217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.345711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.346360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.346643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.346922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.347703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:15.348010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.349475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:15.359054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:15.360221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:15.361222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 298/584] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:15.361830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.361908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 287/585] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:15.362667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.374881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.375544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.466509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21fcc-9daf-4477-b30d-fb129fbe4ec5 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.475078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa9d0-dae6-410f-af0a-a270989c91be None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.499458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.500241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.500554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.500879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.507840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.508498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.508789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.509070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.509914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=2335e75507f44afb8d77f9b70a668a94) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:15.510219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.511287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.511646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.512770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.516960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 288/586] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/role_assignments?user.id=2335e75507f44afb8d77f9b70a668a94&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:15.517723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:15.530440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:15.531427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:15.532672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:15.532672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:15.532672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:15.532672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:15.536644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 299/587] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:15.537597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.607008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.611059 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.642051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe39fa-3cc7-40b1-93e3-03f55406c784 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.653798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab930-e8b3-419d-914d-a070aa19b6e7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.685576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.686373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.686654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.686926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.698329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.699759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.703975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.704765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.705122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.705493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.706418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:15.706740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.708170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:15.720097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:15.721043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:15.721933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:15.721933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:15.721933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:15.721933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:15.727887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 289/588] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:15.727887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.730591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 300/589] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:15.737937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.837081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93524c3-7c37-4d2d-b351-3a87a973f4bf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.850585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1011f88-cde8-4982-b851-0cb9c5752724 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:15.861216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.869508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:15.871440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.872249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.872556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.878800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/de1a294634524a9d8ff48c7c50726228/roles/713299dfe855451c9fd4127a8cca33fc` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.889101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:15.889760 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:15.890040 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:15.890313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:15.890662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=de1a294634524a9d8ff48c7c50726228, role_id=713299dfe855451c9fd4127a8cca33fc)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.891029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:15.891331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:15.892105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.892726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:15.904953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 301/590] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:15.905797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:15.966773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 290/591] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:15 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/de1a294634524a9d8ff48c7c50726228/roles/713299dfe855451c9fd4127a8cca33fc => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:15.967712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.015058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de4cdb-c6d4-4bdb-a6fc-658180b0f1f4 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.064780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.065768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.066227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.066984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.068663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:16.069213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.071439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.094750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 302/592] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:16.094750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.131059 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6331946-184e-40ed-8fec-0b4818b39dc6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.156454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:16.170709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.171583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.171897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.179003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.184543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.186269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:16.212863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:16.213803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:16.215263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:16.215263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:16.215263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:16.215263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:16.215263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 291/593] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:16.215263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.238839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3d119-598f-4f2f-a852-08a5f50e1b8b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.249322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:16.277805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.279438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.279927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.280403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.291351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.293875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:16.313100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:16.314060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 17:30:16.315001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:16.317976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 303/594] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:16.318998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.346134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deb699-7d4e-4374-b787-15bd138d177f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.390646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.391830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.392259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.392845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.401160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.402881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:16.417693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:16.418574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:16.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:16.421782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 292/595] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:16.422993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.434212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:16.444366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cdff9-2417-45ca-9642-74a232f8e9f9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.482630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.483573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.483837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.484193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.485015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:16.485386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.486940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.499508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 304/596] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:16.500521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.520879 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:16.537887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d673d-f6b7-4fcf-9057-6fdb6e537693 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.575515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.576679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.577191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.577743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.578826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:16.579337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.581198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.601215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 293/597] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:16.602335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.617046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677e93d-ba57-47a3-8c8e-39e0e0a99764 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.664253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.664995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.665277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.665549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.666679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:16.667005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.668800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.687294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 305/598] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:16.688154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.710923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:16.719426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64ddc7-4eef-4d4c-ab63-6f18785dfb11 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.759875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.761259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.761944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.762419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.770671 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.772593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:16.788545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:16.789998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:16.790870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:16.793502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 294/599] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:16.794650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.800222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:16.817090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a039c-fed5-4cf2-8655-5d74dc00d1f0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.871724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.871929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.872148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.872311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.885355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.887350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:16.912305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:16.913234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:16.914787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:16.914787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:16.914787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:16.914787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:16.916680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985b292-1ea0-4448-95e9-9a9d0b6632db None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:16.918720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 306/600] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:16.920404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:16.963063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:16.963951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:16.964231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:16.964534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/b6f536917f434773a052aa074d5aa07d/roles/713299dfe855451c9fd4127a8cca33fc` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:16.983030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=b6f536917f434773a052aa074d5aa07d, role_id=713299dfe855451c9fd4127a8cca33fc)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:16.985260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:17.005913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:17.039742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab9902-0839-4018-97cc-dbf79e02d074 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:17.072405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:17.073116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.073401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.073678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.074467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:17.074777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:17.076209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:17.092255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 295/601] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:16 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/b6f536917f434773a052aa074d5aa07d/roles/713299dfe855451c9fd4127a8cca33fc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:17.096323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 307/602] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:17.096696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:17.104521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:17.172606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:17.225148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78138707-12fc-41bb-a49e-191078fdbe37 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:17.233039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57402c-6ebd-419e-909d-6196c6be9b16 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:17.255280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:17.269290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:17.269856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.270129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.270476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/9706ef808a224c83aca165c595b596ef/roles/713299dfe855451c9fd4127a8cca33fc` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.288025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=9706ef808a224c83aca165c595b596ef, role_id=713299dfe855451c9fd4127a8cca33fc)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:17.290243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:17.290373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:17.291216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.291677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.292151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.293244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:17.293872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:17.298174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:17.329559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 296/603] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:17.337835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:17.371634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 308/604] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:17 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/9706ef808a224c83aca165c595b596ef/roles/713299dfe855451c9fd4127a8cca33fc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:17.455035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008044b9-f412-4413-939f-a62e1dff587f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:17.470964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:17.505282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50ea6d-dd31-4d51-a26b-43a0a1314c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:17.506397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50ea6d-dd31-4d51-a26b-43a0a1314c96 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.506830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50ea6d-dd31-4d51-a26b-43a0a1314c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.507805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50ea6d-dd31-4d51-a26b-43a0a1314c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.515563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50ea6d-dd31-4d51-a26b-43a0a1314c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:17.517219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50ea6d-dd31-4d51-a26b-43a0a1314c96 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-ff50ea6d-dd31-4d51-a26b-43a0a1314c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:17.542725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:17.544024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:17.544946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 297/605] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:17.548658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:17.673702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db68eba-31d6-441b-a408-a1333cebfe09 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:17.698087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb7334-8c77-468a-b9fd-a34b9e71c0fa None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.698501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb7334-8c77-468a-b9fd-a34b9e71c0fa None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.698745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb7334-8c77-468a-b9fd-a34b9e71c0fa None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.700345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 298/606] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:17.709974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8f84-c95f-452c-9f8b-e1a6db78f158 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.710235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8f84-c95f-452c-9f8b-e1a6db78f158 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.710990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c8f84-c95f-452c-9f8b-e1a6db78f158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.712881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:17.713544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.713828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.714106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.714901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:17.715210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:17.716712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:17.729789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 309/607] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:17.735574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:17.744464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1e0c8f84-c95f-452c-9f8b-e1a6db78f158 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:17.753871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1e0c8f84-c95f-452c-9f8b-e1a6db78f158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:17.816860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:17.846565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85f6c3-dee7-4f6a-8c4f-8fc0d2e1a864 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:17.910131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 299/608] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:17.910452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:17.911310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:17.911649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:17.911960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/d1d18a38f45e4501914e22689f145fcf/roles/713299dfe855451c9fd4127a8cca33fc` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:17.927636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=d1d18a38f45e4501914e22689f145fcf, role_id=713299dfe855451c9fd4127a8cca33fc)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:17.929199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:17.943248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8f84-c95f-452c-9f8b-e1a6db78f158 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:18.026948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 310/609] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:17 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/d1d18a38f45e4501914e22689f145fcf/roles/713299dfe855451c9fd4127a8cca33fc => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:18.059339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c8f84-c95f-452c-9f8b-e1a6db78f158 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:18.062447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:18.121298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ee022-78b9-489c-8633-995c9bb4a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:18.122527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee022-78b9-489c-8633-995c9bb4a036 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:18.123111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee022-78b9-489c-8633-995c9bb4a036 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:18.123595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ee022-78b9-489c-8633-995c9bb4a036 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:18.151094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ee022-78b9-489c-8633-995c9bb4a036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3d6ee022-78b9-489c-8633-995c9bb4a036 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:18.163732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:18.164861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:18.165698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 300/610] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:18.174264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:18.296577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:18.308994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58755d-7cd9-4487-b105-066ed036b7f0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:18.361873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a343c-d1ef-49f3-84f3-e6f456016514 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:18.362181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a343c-d1ef-49f3-84f3-e6f456016514 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:18.362434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a343c-d1ef-49f3-84f3-e6f456016514 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:18.364026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 301/611] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:18.369449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0914389-5d36-4304-a263-522d9d2612ce None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:18.369992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0914389-5d36-4304-a263-522d9d2612ce None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:18.370484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0914389-5d36-4304-a263-522d9d2612ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:18.371249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:18.371928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:18.372214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:18.372530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:18.373376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:18.373704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:18.375286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:18.389895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b0914389-5d36-4304-a263-522d9d2612ce None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:18.391463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b0914389-5d36-4304-a263-522d9d2612ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:18.398663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 311/612] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:18.410764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:18.474350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 302/613] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:30:18.505294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0914389-5d36-4304-a263-522d9d2612ce None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:18.653901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:18.664891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0914389-5d36-4304-a263-522d9d2612ce None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:18.683449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2e60-32a0-4ac7-b655-f10306de75c4 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:18.736977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:18.738057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:18.738845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:18.739750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:18.744232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:18.745208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:18.748898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:18.749337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:18.761936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:18.770477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:18.771574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:18.772575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:18.778273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 303/614] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:18.784520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:18.784661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:18.785804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:18.813372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:18.814329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:18.815231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:18.815231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:18.815231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:18.815231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:18.822514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 312/615] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:18.826886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:18.832732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:19.008005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a26d9a-4dac-47d5-a6a9-39a854f5f3f1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.040660 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce864-2105-44f8-97ae-c41c6067dcec None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.077466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.078138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.078427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.078706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.079452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:19.079763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.081170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:19.112461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 304/616] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:19.112761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:19.125276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.125956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.128455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.128455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.128455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:19.128455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.128942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:19.150023 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:19.160454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 313/617] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:19.174277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:19.259560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f416fa-fe51-48e3-8aeb-3f198c52d477 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.304034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.306418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.308487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.309069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.329953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.331484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:19.335887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:19.369221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:19.370194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:19.371095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:19.371095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:19.371095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:19.371095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:19.376130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 305/618] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:19.386534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:19.387427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f096626-88fe-4d8b-a16c-a4b1602b40b3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.434999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.436208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.436771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.437051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.448138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.452551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:19.481422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:19.483508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:19.484564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:19.485546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:19.493250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 314/619] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:19.498764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:19.520998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497c2c3-7249-4c9e-b623-8ad60fc72a45 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.558754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.560422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.561515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.562124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.564033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:19.564626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.566838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:19.583652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 306/620] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:19.589902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:19.703728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331903fe-bca5-412a-8f7a-24e992c5d78c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.738447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6af53-4e93-4bc2-aad7-79b566f67087 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.738875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:19.754067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac8fd-e4aa-4e27-a496-bd1bfe0bddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.754757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac8fd-e4aa-4e27-a496-bd1bfe0bddc2 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.755044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac8fd-e4aa-4e27-a496-bd1bfe0bddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.755341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac8fd-e4aa-4e27-a496-bd1bfe0bddc2 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.756479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac8fd-e4aa-4e27-a496-bd1bfe0bddc2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:19.756803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac8fd-e4aa-4e27-a496-bd1bfe0bddc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.758512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac8fd-e4aa-4e27-a496-bd1bfe0bddc2 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:19.774273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:19.781518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 315/621] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jul 20 17:30:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:19.783263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93b62d-5872-4c92-87b4-5bf28a4a7dd2 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.783544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93b62d-5872-4c92-87b4-5bf28a4a7dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.783827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93b62d-5872-4c92-87b4-5bf28a4a7dd2 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.785538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be82b84-0bdd-4134-ac7d-f2897d2459da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.786829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be82b84-0bdd-4134-ac7d-f2897d2459da None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.787280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be82b84-0bdd-4134-ac7d-f2897d2459da None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.787817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be82b84-0bdd-4134-ac7d-f2897d2459da None admin] PATH_INFO: `/v3/users/glance` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.789910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 316/622] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:19.790761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93b62d-5872-4c92-87b4-5bf28a4a7dd2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:19.798266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be82b84-0bdd-4134-ac7d-f2897d2459da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.800011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be82b84-0bdd-4134-ac7d-f2897d2459da None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4be82b84-0bdd-4134-ac7d-f2897d2459da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:19.820290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:19.821553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 17:30:19.822493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:19.831084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 307/623] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:19.831084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49928009-8d45-4ce5-99e5-faf865a93357 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.831084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49928009-8d45-4ce5-99e5-faf865a93357 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.831084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49928009-8d45-4ce5-99e5-faf865a93357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.853600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-49928009-8d45-4ce5-99e5-faf865a93357 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:19.855382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-49928009-8d45-4ce5-99e5-faf865a93357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:19.924502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93b62d-5872-4c92-87b4-5bf28a4a7dd2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:19.934680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 308/624] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:19.935355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49928009-8d45-4ce5-99e5-faf865a93357 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:19.970984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c45a53-b779-4d24-a1dd-70ad781b6b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:19.971916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c45a53-b779-4d24-a1dd-70ad781b6b69 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:19.972270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c45a53-b779-4d24-a1dd-70ad781b6b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:19.972652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c45a53-b779-4d24-a1dd-70ad781b6b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:19.983781 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c45a53-b779-4d24-a1dd-70ad781b6b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:19.985975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c45a53-b779-4d24-a1dd-70ad781b6b69 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-00c45a53-b779-4d24-a1dd-70ad781b6b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:20.005390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:20.006338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:20.007342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:20.007342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:20.007342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:20.007342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:20.007342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 317/625] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:20.008063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a68c9-661b-4b1b-aecf-ece8c240fe98 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.008063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a68c9-661b-4b1b-aecf-ece8c240fe98 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.008242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a68c9-661b-4b1b-aecf-ece8c240fe98 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.009639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 318/626] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:20.010171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a68c9-661b-4b1b-aecf-ece8c240fe98 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.042833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.053294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49928009-8d45-4ce5-99e5-faf865a93357 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.073348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.090227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.090992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.091284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.091726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.092655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:20.095807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:20.095807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.124624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 309/627] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jul 20 17:30:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:20.125700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.128785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a68c9-661b-4b1b-aecf-ece8c240fe98 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.170481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f034aa2-e602-47d5-a6e6-45523f6f7769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.171285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034aa2-e602-47d5-a6e6-45523f6f7769 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.171577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034aa2-e602-47d5-a6e6-45523f6f7769 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.171853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f034aa2-e602-47d5-a6e6-45523f6f7769 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.182678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f034aa2-e602-47d5-a6e6-45523f6f7769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-4f034aa2-e602-47d5-a6e6-45523f6f7769 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:20.190300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:20.191275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:20.192132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 319/628] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:20.192902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665bbd-0e7f-4d36-89fa-3f14f6541cc1 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.192974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665bbd-0e7f-4d36-89fa-3f14f6541cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.193824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665bbd-0e7f-4d36-89fa-3f14f6541cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.211414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a0665bbd-0e7f-4d36-89fa-3f14f6541cc1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:20.213030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a0665bbd-0e7f-4d36-89fa-3f14f6541cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:20.238810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c326e0-0dc4-4823-afdc-0d422f268437 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.282179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.283166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.283589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.284011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.284798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:20.285116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:20.286538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.287527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 320/629] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:20.288219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665bbd-0e7f-4d36-89fa-3f14f6541cc1 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.304865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 310/630] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:20.305636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.318252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.339799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.391446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665bbd-0e7f-4d36-89fa-3f14f6541cc1 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.402821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afe918-50c9-42f0-ac52-c8bb5a56b3f5 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.424854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.425487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.425767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.426036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.437608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f387e4d-a302-4168-a56f-6c384c304c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.438270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f387e4d-a302-4168-a56f-6c384c304c7a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.438541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f387e4d-a302-4168-a56f-6c384c304c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.438821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f387e4d-a302-4168-a56f-6c384c304c7a None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.439561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f387e4d-a302-4168-a56f-6c384c304c7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:20.439868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f387e4d-a302-4168-a56f-6c384c304c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:20.441261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f387e4d-a302-4168-a56f-6c384c304c7a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.449133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 311/631] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:20.450381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b2177-605a-4c36-a1c2-323d0be21ec1 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.450671 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b2177-605a-4c36-a1c2-323d0be21ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.450946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b2177-605a-4c36-a1c2-323d0be21ec1 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.452156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 312/632] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:20.452859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b2177-605a-4c36-a1c2-323d0be21ec1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.498933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:20.500383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.522997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.549614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.554554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b2177-605a-4c36-a1c2-323d0be21ec1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.587990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.588838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.589125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.589406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.596008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:20.597333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 321/633] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:20.597474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.598151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:20.615422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:20.617076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 313/634] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:20.618011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.700673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a713753-7819-4930-b9e7-ec6f61efc787 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.720021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0b1fe-4301-40b0-9ec7-26e225023a45 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.725868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.737829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a6182-cd23-43fe-830c-04e501fa5ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.738620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a6182-cd23-43fe-830c-04e501fa5ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.738899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a6182-cd23-43fe-830c-04e501fa5ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.742833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a6182-cd23-43fe-830c-04e501fa5ce1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.749307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a6182-cd23-43fe-830c-04e501fa5ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:20.753666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.754451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.754736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.755012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-c40a6182-cd23-43fe-830c-04e501fa5ce1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:20.757034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:20.757956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:20.758821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 322/635] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:20.759992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68de14-ccd8-4567-89ee-3564771fa07d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.760089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68de14-ccd8-4567-89ee-3564771fa07d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.762167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68de14-ccd8-4567-89ee-3564771fa07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.768019 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:20.769513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.776905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4c68de14-ccd8-4567-89ee-3564771fa07d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:20.778075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4c68de14-ccd8-4567-89ee-3564771fa07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:20.787318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:20.789142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:20.790051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:20.790051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:20.790051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:20.790051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:20.790051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 314/636] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:20.790051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.803322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.847405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 323/637] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:20.848171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68de14-ccd8-4567-89ee-3564771fa07d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.909895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2fa78-8f31-4bee-9da1-09d31a58053b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.944538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:20.945854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:20.946146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:20.946425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:20.955118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68de14-ccd8-4567-89ee-3564771fa07d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:20.957525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:20.958992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:20.968163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:20.977458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:20.978425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:20.979356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:20.979356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:20.979356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:20.979356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:20.979356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 315/638] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:20.979356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:20.997533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.003561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.003561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.003561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.003561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:21.003561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.005969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.020731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 324/639] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:21.021589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.045929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.083001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d561ed9-73dc-485c-ad91-d1c90e62f7f1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.116466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.117161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.117453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.117737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.118487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:21.118802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.120168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.121316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc3101-8cc8-4676-9e31-e7aa37fad081 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.128049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 316/640] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:21.128868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.154297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.154953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.155233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.155508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.156217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:21.156620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.158235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.169320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 325/641] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:21.170108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.229301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c132c-e392-4ef3-b548-482d42183195 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.260946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.261723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.262006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.262421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.263857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91685-9903-4fe9-8c7a-76144c34f27f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.272543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:21.280613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:21.280849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:21.282265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 317/642] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:21.283120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.302762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.303431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.303714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.304534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.305276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:21.305587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.306993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.318623 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 326/643] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:21.319209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.319405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.383650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4df552-812c-43d4-b1e5-6d3a9d4282cb None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.409060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.417573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.418131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f416c8f-3386-441d-ad6a-4118c47138e1 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.421122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.421122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.421122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/2335e75507f44afb8d77f9b70a668a94/roles/713299dfe855451c9fd4127a8cca33fc` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.433476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=2335e75507f44afb8d77f9b70a668a94, role_id=713299dfe855451c9fd4127a8cca33fc)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.434922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.451153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.451945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.452232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.452540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.462690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.464125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:21.481696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:21.482743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:21.483952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:21.483952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:21.483952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:21.483952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:21.484583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 327/644] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:21.485439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.500337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 318/645] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:21 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/2335e75507f44afb8d77f9b70a668a94/roles/713299dfe855451c9fd4127a8cca33fc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:21.501574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.540786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.613030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1f252-7bb8-4270-a87f-74190a237c41 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.629385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ba0a-990c-4595-8708-8358e8cfccf7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.650726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.652088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.652088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.652088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.664762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.666221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.671643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.672312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.672626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.672903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.673651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:21.673961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.675313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:21.682522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:21.683525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:21.684939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 328/646] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:21.685579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 319/647] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:21.685670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.685753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.696869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.787514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.794325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277851b5-6cec-4ede-b32f-edf9e8088bc9 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.809367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26202767-7446-4c2b-87a3-ea9df88d2aa0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.836155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.836879 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.837193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.837444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.838174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:21.838485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.839866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.845892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:21.847151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:21.847432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:21.847705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:21.855798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 320/648] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:21.855798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.856062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:21.857114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:21.872798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:21.873737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:21.874643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:21.877081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 329/649] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:21.878099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:21.951356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:21.970176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e53c1-f243-404e-a6e9-7fb319dda3d7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:21.992219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1874e-a0dd-4a69-bcf7-17450b89e133 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.007873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.008712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.009000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.009563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.025105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.026553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.031744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.032704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.033030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.033320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.034157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:22.034476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.035913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:22.046524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:22.047447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:22.048575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:22.048575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:22.048575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:22.048575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:22.054841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 321/650] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:22.054841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.055068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 330/651] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:30:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:22.055153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.068969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:22.166236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d001fcd-a2d1-4ec2-b400-77243929fa2d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.169079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51522-1637-4174-ac17-6f24da51c116 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.190290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:22.203105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.203792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.204150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.204526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.205327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:22.205643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.207090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.207986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.209036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.209579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.209784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.217133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.218568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.219005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 331/652] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:22.219931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:22.243312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:22.244511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 322/653] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:22.245517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.320961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:22.336974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4294e-f7f6-4fbe-80bd-331acf7b0481 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.360138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151f3a3-58b8-441b-902c-955585df56a7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.377270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.377937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.378230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.378510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.379253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:22.379564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.380984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.396245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.396949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.397238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.397519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.398237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 332/654] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:22.399129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.445641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:22.470393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.471800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.507410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3f0d-6138-4deb-97ca-1f1108b63eea None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.540811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:22.545440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.547137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.547137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.547137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.548416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=de1a294634524a9d8ff48c7c50726228) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:22.548416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.550203 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.564873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 333/655] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/role_assignments?user.id=de1a294634524a9d8ff48c7c50726228&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:22.566804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.581445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 323/656] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:22.582829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.714783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167eb660-76bc-40f5-81a1-1ffa1d91415b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.719009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed8eba-7ea8-40d3-96dd-1426f734b428 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.729550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:22.764020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.764368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.764611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.764892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.772798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.773795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.774152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.774522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.776691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:22.776691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.776883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.777673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.777844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:22.793260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:22.794632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 324/657] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:22.795965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.804576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 334/658] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:22.805560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:22.831726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:22.919989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bc962-285b-4c2a-8395-e63a4bd03bc2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.959461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:22.960931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:22.961283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:22.961624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:22.962787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600ff5-ea23-4490-bbbe-fb42c22d77b8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:22.973417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:22.974477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:22.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:22.997001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:22.998133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:22.998133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:22.998133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:22.998133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:23.000407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 325/659] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:23.009527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.012151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.013098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.013551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.013986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.014993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:23.015817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.018896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:23.041291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 335/660] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Jul 20 17:30:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:23.042476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.069114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.128621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.150295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458d6d5-70ff-43e7-83dc-eebb2406c5d6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:23.166626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee4fa-20b7-468a-86d6-e6eb4296add9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:23.187175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.187897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.188278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.188691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.209130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.209949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.210451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.210822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.211653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:23.212057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.213621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:23.236414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 336/661] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:23.237814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.276705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.278451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:23.325920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.351927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de738db-bd60-4096-b88d-6d4713b21a26 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:23.396351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.405700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6f63-d681-4f2f-9b7a-d53d846616a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.407230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6f63-d681-4f2f-9b7a-d53d846616a1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.407716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6f63-d681-4f2f-9b7a-d53d846616a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.408310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6f63-d681-4f2f-9b7a-d53d846616a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.420353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 326/662] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:23.421657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.511861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6f63-d681-4f2f-9b7a-d53d846616a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.513664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6f63-d681-4f2f-9b7a-d53d846616a1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:23.551135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47786d-e1eb-4291-b322-64f082214a10 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:23.594638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.595519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.595872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.596229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.603505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.605276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:23.611633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:23.619423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:23.620338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:23.621282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 327/663] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:23.621789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.636251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 337/664] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:23.637886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e11b57-a53c-4f64-a61b-119ccbea0010 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.638401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e11b57-a53c-4f64-a61b-119ccbea0010 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.638879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e11b57-a53c-4f64-a61b-119ccbea0010 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.640348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 338/665] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:23.641497 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e11b57-a53c-4f64-a61b-119ccbea0010 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.671912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.735741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c17bf-ea00-48e2-ad25-9a52b81ab88f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:23.761664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e11b57-a53c-4f64-a61b-119ccbea0010 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:23.772298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.773645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.773928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.775319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.786934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.788483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:23.800763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf83409-855f-4fb2-b8d4-5df4dfec0d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.801469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83409-855f-4fb2-b8d4-5df4dfec0d44 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.801703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83409-855f-4fb2-b8d4-5df4dfec0d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.801973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf83409-855f-4fb2-b8d4-5df4dfec0d44 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.802758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf83409-855f-4fb2-b8d4-5df4dfec0d44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:23.803065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf83409-855f-4fb2-b8d4-5df4dfec0d44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.805043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf83409-855f-4fb2-b8d4-5df4dfec0d44 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:23.806488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:23.807820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:23.808906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:23.808906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:23.808906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:23.808906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:23.812347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 328/666] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:23.813215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.823822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 339/667] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:23.825296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d080d-437c-444b-a4bb-b5326d5d8f60 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.825582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d080d-437c-444b-a4bb-b5326d5d8f60 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.825856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d080d-437c-444b-a4bb-b5326d5d8f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.843011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-878d080d-437c-444b-a4bb-b5326d5d8f60 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:23.844250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-878d080d-437c-444b-a4bb-b5326d5d8f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:23.882575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.914700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 340/668] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:23.915506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d080d-437c-444b-a4bb-b5326d5d8f60 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:23.931709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:23.943519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf94752-0d41-4e22-9af4-52a080743138 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:23.980021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:23.980838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:23.981125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:23.981406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:23.993604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:23.995060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:24.012809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:24.019537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:24.020496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:24.020496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:24.020496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:24.020496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:24.020496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 329/669] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:24.020496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.020852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d080d-437c-444b-a4bb-b5326d5d8f60 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.062697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.063292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.063573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.064002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.065108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:24.065426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.067225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.086560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 341/670] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:24.086560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.125465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a38e6a-9ede-40b6-9334-8adc933e2f5c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.126271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:24.162159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.162857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.163129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.164020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.164895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=b6f536917f434773a052aa074d5aa07d) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:24.165218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.166656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.172830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 330/671] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/role_assignments?user.id=b6f536917f434773a052aa074d5aa07d&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:24.173415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.185365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:24.217056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d1d65-ffd9-422c-a2f4-c25102c58ce7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.262799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.263627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.263913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.264193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.278316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:24.286664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:24.287696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:24.288626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:24.292918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 342/672] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:24.294281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.312027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24190f6-eea4-4d3b-8e1b-4656165e6c74 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.365703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.365703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.366281 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.368571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.369136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:24.369424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.371276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.385810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 331/673] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:24.386492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.431130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:24.444752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c9322-31c3-4e48-b0e5-d9d055bd8caf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.478853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:24.492400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.493544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.493835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.494132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.502847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.504845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.517612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90729936-5e45-449b-b47b-e79c107d4336 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:24.524368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:24.525311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:24.526233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:24.530656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 343/674] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:24.531579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.561913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.562983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.563475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.563956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.564949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:24.565463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.567175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.576851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 332/675] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:24.582393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.659034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c3390-5626-4278-9480-c972886c4b71 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.694897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58f15-995f-4283-ba49-1ed3a51a6c0d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.699934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.700901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.701188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.701459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.708407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.709831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.726910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:24.727205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:24.728107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:24.729065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 344/676] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:24.729997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.737554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.738556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.739173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.745914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.745914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:24.745914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.745914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.748841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:24.762105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 333/677] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:24.763348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.845216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2cd97-4989-4ff6-b8b1-771f5e339c89 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.881599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e29d6e-bf29-46ce-8655-261abaa9c609 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:24.885355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.886260 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.886651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.887030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.897978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.899531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.919226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61329bcf-f21a-4589-bef8-fd840309b578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:24.919887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61329bcf-f21a-4589-bef8-fd840309b578 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.921106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61329bcf-f21a-4589-bef8-fd840309b578 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.921106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61329bcf-f21a-4589-bef8-fd840309b578 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:24.921327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:24.922386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:24.923494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:24.923494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:24.923494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:24.923494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:24.923764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61329bcf-f21a-4589-bef8-fd840309b578 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:24.923764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61329bcf-f21a-4589-bef8-fd840309b578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:24.923764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61329bcf-f21a-4589-bef8-fd840309b578 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:24.929521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 345/678] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:24.930496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:24.940485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 334/679] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:24.940485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec1ac5-389d-4669-8af1-fe9c8b80626f None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:24.940485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec1ac5-389d-4669-8af1-fe9c8b80626f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:24.940485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec1ac5-389d-4669-8af1-fe9c8b80626f None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:24.940485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 335/680] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:24.940915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec1ac5-389d-4669-8af1-fe9c8b80626f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:25.002646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:25.005065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:25.048220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c33a3d-ccd8-44f8-acda-9f54b9e1c368 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.059499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec1ac5-389d-4669-8af1-fe9c8b80626f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.089700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.090494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.090871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.091380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.092198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=9706ef808a224c83aca165c595b596ef) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:25.092588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:25.096220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:25.099518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f6ad6-6300-4a72-a218-679c6c06e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.100255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f6ad6-6300-4a72-a218-679c6c06e1fd None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.100742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f6ad6-6300-4a72-a218-679c6c06e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.101681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f6ad6-6300-4a72-a218-679c6c06e1fd None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.102516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f6ad6-6300-4a72-a218-679c6c06e1fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:25.102930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f6ad6-6300-4a72-a218-679c6c06e1fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:25.104385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f6ad6-6300-4a72-a218-679c6c06e1fd None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:25.107333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 346/681] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/role_assignments?user.id=9706ef808a224c83aca165c595b596ef&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:25.107983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:25.130170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 336/682] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:25.132301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d6ce1-75ad-433c-aee8-da8149e330de None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.132876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d6ce1-75ad-433c-aee8-da8149e330de None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.133399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d6ce1-75ad-433c-aee8-da8149e330de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.156800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-8c6d6ce1-75ad-433c-aee8-da8149e330de None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:25.159179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8c6d6ce1-75ad-433c-aee8-da8149e330de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:25.235900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea3973-c63c-4eac-a5f3-d986279e6cfb None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.254608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 337/683] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:25.256161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d6ce1-75ad-433c-aee8-da8149e330de None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:25.265035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:25.275420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.276080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.276361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.279286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.279286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=d1d18a38f45e4501914e22689f145fcf) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:25.279286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:25.279526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:25.288144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 347/684] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:25 2022] GET /identity/v3/role_assignments?user.id=d1d18a38f45e4501914e22689f145fcf&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:25.290771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:25.320414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:25.396495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d6ce1-75ad-433c-aee8-da8149e330de None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.449049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bb81b-0dd3-4ae6-8500-ae675549eb62 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.451514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.452383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.453636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.453636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.463471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:25.465245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:25.486504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:25.487476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:25.488661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:25.491116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 338/685] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:25.495848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:25.515513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.517258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.521118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.521411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.535401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:25.538021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:25.562077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:25.563038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:25.563939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:25.563939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:25.563939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:25.563939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:25.568798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 348/686] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:25.571587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:25.640642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:25.644714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ac150-582e-403c-a240-91e02ca3aff1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.654355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:25.684093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.685048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.685611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.686058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.687052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:25.687502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:25.689278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:25.709581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 339/687] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jul 20 17:30:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:25.715441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:25.826433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7273-5c1f-43ec-b188-39c9ebc9435f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.852058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b56b86-113c-4ded-897b-b77680bea749 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:25.876127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260355b-4802-494b-898e-27e768baceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.876829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260355b-4802-494b-898e-27e768baceaf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.877111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260355b-4802-494b-898e-27e768baceaf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.877385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260355b-4802-494b-898e-27e768baceaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.928049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:25.929492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:25.930158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:25.930444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:25.930716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:25.931442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:25.931735 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:25.933128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:25.959220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 349/688] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:25.969703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:26.008499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260355b-4802-494b-898e-27e768baceaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:26.011511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260355b-4802-494b-898e-27e768baceaf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:26.162116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f621eb-ffff-422c-b3a9-f446b5938a27 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:26.167809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:26.220665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 340/689] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:26.226426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260355b-4802-494b-898e-27e768baceaf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:26.239070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:26.239751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:26.240036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:26.240315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:26.242990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:26.244473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:26.306283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:26.338743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 350/690] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:30:25 2022] POST /identity/v3/users => generated 254 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:26.372184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260355b-4802-494b-898e-27e768baceaf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:26.416285 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aafd2-e896-472f-ba0a-606c1e812ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:26.417259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aafd2-e896-472f-ba0a-606c1e812ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:26.417584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aafd2-e896-472f-ba0a-606c1e812ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:26.417895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aafd2-e896-472f-ba0a-606c1e812ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:26.449445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aafd2-e896-472f-ba0a-606c1e812ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:26.451275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aafd2-e896-472f-ba0a-606c1e812ad8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-b01aafd2-e896-472f-ba0a-606c1e812ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:26.476695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:26.477727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:26.478630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:26.478630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:26.478630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:26.478630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:26.481079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 341/691] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:26.489914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:26.503858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039fb0c-7e02-4f46-8794-e5b9ea7caac8 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:26.504190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039fb0c-7e02-4f46-8794-e5b9ea7caac8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:26.504452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039fb0c-7e02-4f46-8794-e5b9ea7caac8 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:26.506606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 342/692] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:30:26.513515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af594c-c720-430e-ae70-a71852abd674 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:26.513818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af594c-c720-430e-ae70-a71852abd674 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:26.514039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af594c-c720-430e-ae70-a71852abd674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:26.561067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-79af594c-c720-430e-ae70-a71852abd674 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:26.563046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-79af594c-c720-430e-ae70-a71852abd674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:26.635223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ff796-b60f-4fbe-b7d2-9979a9a9de6d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:26.664465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 343/693] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 17:30:26.683017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af594c-c720-430e-ae70-a71852abd674 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:26.688611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:26.689526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:26.690050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:26.690513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:26.691933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:26.692443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:26.694478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:26.715692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 351/694] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:26.721422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:26.756816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:26.784680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:26.903530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af594c-c720-430e-ae70-a71852abd674 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:26.921299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244871f-19e5-40b6-9292-ae481d20eb89 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:26.972956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:26.973766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:26.974052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:26.974338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:26.987516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:26.993705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:26.994525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:26.994811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:26.995087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:27.002142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:27.003068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:27.003918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:27.006798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 344/695] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:27.013826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:27.021334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:27.023158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:27.046702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:27.047672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:27.048628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:27.061475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 352/696] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:27.062624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:27.199854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:27.222763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4b61f-42d1-4236-b928-124bc1dff8d0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:27.244825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:27.249735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ceb18-7ca4-491f-819d-cfce251b8547 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:27.297143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:27.297825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.298119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.298407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.299172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:27.299487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:27.301354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:27.311017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 345/697] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:27.316531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:27.321156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:27.321837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.322120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.322394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.323194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:27.323502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:27.324979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:27.353265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 353/698] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:27.358043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:27.423291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56bd6-7da2-48a8-9b08-c03d958a80d7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:27.477436 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:27.478162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.478451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.478734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.496828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:27.498309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:27.530324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:27.531280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:27.532475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:27.532475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:27.532475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:27.532475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:27.533126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 346/699] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:27.546208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:27.595887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:27.604869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c30aa-dc3e-4138-81c8-6e815cde4f13 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:27.657667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416c64-fad2-4095-9546-dc3d5683ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:27.658341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416c64-fad2-4095-9546-dc3d5683ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.658683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416c64-fad2-4095-9546-dc3d5683ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.658963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416c64-fad2-4095-9546-dc3d5683ffd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.660789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416c64-fad2-4095-9546-dc3d5683ffd1 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=2335e75507f44afb8d77f9b70a668a94) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:27.661137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416c64-fad2-4095-9546-dc3d5683ffd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:27.662605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416c64-fad2-4095-9546-dc3d5683ffd1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:27.669759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 354/700] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/role_assignments?user.id=2335e75507f44afb8d77f9b70a668a94&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:27.671079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbb3d-2104-4463-b05e-c7ccc9814cf4 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.671369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbb3d-2104-4463-b05e-c7ccc9814cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.671643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fbb3d-2104-4463-b05e-c7ccc9814cf4 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.677125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 355/701] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:27.680462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cff8eb-2e8d-47d6-9655-33c837ed8143 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.680648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cff8eb-2e8d-47d6-9655-33c837ed8143 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.680837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cff8eb-2e8d-47d6-9655-33c837ed8143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.703239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:27.716591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f0252-253f-4407-acc4-e79a592419e7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:27.729676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-79cff8eb-2e8d-47d6-9655-33c837ed8143 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:27.730823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-79cff8eb-2e8d-47d6-9655-33c837ed8143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:27.755860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3bf67-813a-4679-9fa0-67c50eb4c9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:27.756742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3bf67-813a-4679-9fa0-67c50eb4c9aa None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.757071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3bf67-813a-4679-9fa0-67c50eb4c9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.757356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3bf67-813a-4679-9fa0-67c50eb4c9aa None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.758164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3bf67-813a-4679-9fa0-67c50eb4c9aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:27.758478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3bf67-813a-4679-9fa0-67c50eb4c9aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:27.760032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3bf67-813a-4679-9fa0-67c50eb4c9aa None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:27.793328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 347/702] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:27.796463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0a8e-16d9-4e4a-b5c6-16d73fca965b None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.796463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0a8e-16d9-4e4a-b5c6-16d73fca965b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.796463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b0a8e-16d9-4e4a-b5c6-16d73fca965b None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.804752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 348/703] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:27 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:27.808460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0a8e-16d9-4e4a-b5c6-16d73fca965b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:27.824585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 356/704] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:27.826133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb5eb-0a42-4e7c-8ed6-f2f603331963 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.826425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb5eb-0a42-4e7c-8ed6-f2f603331963 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.832481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bb5eb-0a42-4e7c-8ed6-f2f603331963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.858661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1e2bb5eb-0a42-4e7c-8ed6-f2f603331963 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:27.864460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1e2bb5eb-0a42-4e7c-8ed6-f2f603331963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:27.902898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:27.926918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b0a8e-16d9-4e4a-b5c6-16d73fca965b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:27.948423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 357/705] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:27.953568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb5eb-0a42-4e7c-8ed6-f2f603331963 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:27.964511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:27.965309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:27.965596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:27.965874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:27.974603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:27.976127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:27.991090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:27.992131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:27.993175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:27.996520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 349/706] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:27.997420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.038272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:28.078561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bb5eb-0a42-4e7c-8ed6-f2f603331963 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.116854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ef92a-3750-4aa2-b29a-ff11fa4268f1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.134381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:28.135339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:28.135628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:28.136169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:28.137250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:28.138650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:28.155061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:28.156037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:28.156424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:28.156833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:28.157521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:28.158662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:28.159902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 358/707] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:28.161078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:28.161162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.164611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:28.175951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:28.195134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:28.196166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:28.197036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:28.198488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 350/708] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:27 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:28.198952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.273962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077924e-b262-4db3-a11f-92bc400f2880 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.306309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:28.317201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:28.325144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74637a5-ed12-4a85-a501-60c3070042e2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.325284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:28.325284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:28.325284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:28.325284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:28.325284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:28.325284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:28.352148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 359/709] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:30:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:28.353473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.361065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:28.361737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:28.362024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:28.362306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:28.363028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:28.363339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:28.364777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:28.374690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 351/710] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jul 20 17:30:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:28.374690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.455611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:28.474563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41dbe-b66c-4297-a74b-ae29806fdde5 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.504154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380a660-7b34-4da8-8eaa-726eccbcd91c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.521327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:28.521959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:28.522316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:28.522720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:28.523506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:28.523886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:28.525406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:28.540464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 360/711] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:28.540464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.552475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:28.555362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:28.559631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:28.560170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:28.604930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:28.653696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:28.655526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:28.660455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f07db-67b9-4db3-97d2-ff0e4db295f0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.713252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:28.713848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:28.714128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:28.714402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:28.715720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:28.715720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:28.717400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:28.728831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 361/712] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jul 20 17:30:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:28.729824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.745569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:28.794442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 352/713] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:28.804891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:28.969043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50835d-db82-492b-9150-3c166ac927aa None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:28.987420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:28.994499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a3a70-6615-423c-a16e-0ec23e9faae8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:29.033733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42da08f-57a8-4ce4-8700-a8c681a610e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:29.034410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da08f-57a8-4ce4-8700-a8c681a610e7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.034985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da08f-57a8-4ce4-8700-a8c681a610e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.035163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da08f-57a8-4ce4-8700-a8c681a610e7 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.036700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42da08f-57a8-4ce4-8700-a8c681a610e7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:29.037179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42da08f-57a8-4ce4-8700-a8c681a610e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:29.040042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42da08f-57a8-4ce4-8700-a8c681a610e7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:29.049034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 362/714] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:29.050301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cf44-c3b8-4dea-b56c-dede0994177b None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.050586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cf44-c3b8-4dea-b56c-dede0994177b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.051888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cf44-c3b8-4dea-b56c-dede0994177b None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.056366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 363/715] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:29.057780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:29.058550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.058833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.059109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.066284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169d29a-7be6-47e8-aeaa-281b3ac9e98b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.066598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169d29a-7be6-47e8-aeaa-281b3ac9e98b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.066849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169d29a-7be6-47e8-aeaa-281b3ac9e98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.088517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:29.089996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:29.095883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7169d29a-7be6-47e8-aeaa-281b3ac9e98b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:29.097162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7169d29a-7be6-47e8-aeaa-281b3ac9e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:29.125737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:29.126751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:29.127677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:29.127677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:29.127677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:29.127677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:29.134929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 353/716] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:29.135884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:29.176569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:29.275253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 364/717] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:29.307040 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169d29a-7be6-47e8-aeaa-281b3ac9e98b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:29.335134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b470ac-db86-427a-9e05-824e74b6cc22 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:29.389482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:29.390613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.391203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.391710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.392719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:29.393276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:29.395029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:29.407936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 354/718] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:29.415319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:29.499730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:29.528202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d81b36-53af-4dbe-bed8-921300f7f9e6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:29.574006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:29.575168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.575827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.576310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.584228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:29.586573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:29.604265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:29.605243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:29.606721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:29.611080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 355/719] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:29.618477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:29.644405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169d29a-7be6-47e8-aeaa-281b3ac9e98b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:29.664510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:29.734083 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ddb90-a7ca-47c2-b267-4b79934092c8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:29.761192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:29.762024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.762587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.786572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:29.787641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:29.788334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:29.788919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:29.789916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:29.790740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:29.794102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:29.796098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:29.800058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:29.812547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:29.825017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:29.825935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 365/720] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 511 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:29.828092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:29.830747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 356/721] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:29.838919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:30.025700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2e079-7d6b-4f74-bae5-72872ef8cbac None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:30.046391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9636d-372f-4488-ae84-f110188c5bd7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:30.077851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:30.078843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.079320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.079771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:30.080726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:30.081211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:30.082775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:30.104550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 366/722] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:30.110235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:30.160362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72c823-1b0a-4e7f-a162-0e6f091d34f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:30.161058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72c823-1b0a-4e7f-a162-0e6f091d34f6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.161346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72c823-1b0a-4e7f-a162-0e6f091d34f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.163742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72c823-1b0a-4e7f-a162-0e6f091d34f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:30.163742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72c823-1b0a-4e7f-a162-0e6f091d34f6 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=b6f536917f434773a052aa074d5aa07d) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:30.163742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72c823-1b0a-4e7f-a162-0e6f091d34f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:30.164241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72c823-1b0a-4e7f-a162-0e6f091d34f6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:30.173853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:30.178047 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 357/723] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:29 2022] GET /identity/v3/role_assignments?user.id=b6f536917f434773a052aa074d5aa07d&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:30.274136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1cf37-f65f-48aa-b000-5689552d7c72 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.274604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1cf37-f65f-48aa-b000-5689552d7c72 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.275013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1cf37-f65f-48aa-b000-5689552d7c72 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:30.277120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 358/724] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:30:30.295190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac3fd8-8e80-420f-8d98-02049fff3862 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.295633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac3fd8-8e80-420f-8d98-02049fff3862 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.296019 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac3fd8-8e80-420f-8d98-02049fff3862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:30.328544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd8a28-0e5e-4488-8bec-b93e47df98c6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:30.341654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e8ac3fd8-8e80-420f-8d98-02049fff3862 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:30.343499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e8ac3fd8-8e80-420f-8d98-02049fff3862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:30.383682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:30.384784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.385216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.385657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:30.408593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:30.410374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:30.418074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:30.440527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:30.441577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:30.442463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:30.442463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:30.442463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:30.442463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:30.442463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 367/725] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:30.452907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:30.455673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 359/726] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:30.530208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac3fd8-8e80-420f-8d98-02049fff3862 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:30.600662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:30.646132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe630bd-dcdb-49ba-ad81-5709227e5b82 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:30.715594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:30.716610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.717090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.717725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:30.719054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:30.719602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:30.721731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:30.747886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 368/727] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:30.752898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:30.798666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac3fd8-8e80-420f-8d98-02049fff3862 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:30.885229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:30.885980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.886272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.886546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:30.899249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64a0cc-6ebd-47d0-b584-9f80deadd805 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:30.918252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:30.938874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:30.940125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:30.941071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 360/728] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:30.948765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:30.971594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:30.972364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:30.972674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:30.972951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.002362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:31.005482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:31.046381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:31.047379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:31.048576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 369/729] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:30:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:31.054408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:31.060459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:31.078675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:31.190963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f12e0-1e94-4e15-897e-1e0e5b304fd0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:31.274852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8b2f5-e9ff-429a-a006-8ca25320dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:31.275538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8b2f5-e9ff-429a-a006-8ca25320dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:31.275827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8b2f5-e9ff-429a-a006-8ca25320dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:31.276109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8b2f5-e9ff-429a-a006-8ca25320dfa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.276908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8b2f5-e9ff-429a-a006-8ca25320dfa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:31.277222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8b2f5-e9ff-429a-a006-8ca25320dfa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:31.278631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8b2f5-e9ff-429a-a006-8ca25320dfa6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:31.294111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782fb06-852e-4d56-8107-509f1051b4d9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:31.299145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 361/730] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:31.300995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8c08c-1a94-4de8-94bd-89b01f2b9b8c None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:31.304407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8c08c-1a94-4de8-94bd-89b01f2b9b8c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:31.304981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8c08c-1a94-4de8-94bd-89b01f2b9b8c None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.307225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 362/731] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:31.308154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8c08c-1a94-4de8-94bd-89b01f2b9b8c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:31.342485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeda41e-5653-4f1a-9c44-b4042c0f6c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:31.343171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda41e-5653-4f1a-9c44-b4042c0f6c74 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:31.343459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda41e-5653-4f1a-9c44-b4042c0f6c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:31.343738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda41e-5653-4f1a-9c44-b4042c0f6c74 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.345057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda41e-5653-4f1a-9c44-b4042c0f6c74 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:31.345388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda41e-5653-4f1a-9c44-b4042c0f6c74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:31.347125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda41e-5653-4f1a-9c44-b4042c0f6c74 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:31.370346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 370/732] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jul 20 17:30:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:31.371885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ace24-ae6a-4c46-b16e-3bb0c45a05c4 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:31.372183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ace24-ae6a-4c46-b16e-3bb0c45a05c4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:31.372648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ace24-ae6a-4c46-b16e-3bb0c45a05c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.393106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d66ace24-ae6a-4c46-b16e-3bb0c45a05c4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:31.397017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d66ace24-ae6a-4c46-b16e-3bb0c45a05c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:31.476929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:31.488255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 371/733] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:31.489321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ace24-ae6a-4c46-b16e-3bb0c45a05c4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:31.551829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8c08c-1a94-4de8-94bd-89b01f2b9b8c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:31.589237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:31.624994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:31.627254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:31.627566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:31.627856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.652235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:31.653753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:31.693924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:31.694914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:31.695797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:31.695797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:31.695797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:31.695797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:31.695797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 363/734] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:31.696631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:31.710460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ace24-ae6a-4c46-b16e-3bb0c45a05c4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:31.752811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:31.753998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:31.754484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:31.754959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.828975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:31.840692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:31.842225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:31.859227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24311b78-2cf5-460f-ab76-92d01eb1b2a4 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:31.908308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd61382-01d9-4237-8cbb-9717495cb3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:31.909585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd61382-01d9-4237-8cbb-9717495cb3a8 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:31.909970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd61382-01d9-4237-8cbb-9717495cb3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:31.910376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd61382-01d9-4237-8cbb-9717495cb3a8 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:31.911271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd61382-01d9-4237-8cbb-9717495cb3a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:31.914146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd61382-01d9-4237-8cbb-9717495cb3a8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:31.967797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 372/735] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 17:30:31.969139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.026989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 364/736] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:30:31 2022] POST /identity/v3/services => generated 251 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:32.026989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f66d-2522-49b4-9078-af8caf5ea2f4 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.026989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f66d-2522-49b4-9078-af8caf5ea2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.026989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f66d-2522-49b4-9078-af8caf5ea2f4 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.026989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 365/737] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:32.026989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0f66d-2522-49b4-9078-af8caf5ea2f4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.069131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:32.104286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:32.118729 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a0080-3a86-4150-9f09-d3686a7e31af None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:32.177782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f627e8a-b406-4f22-bba1-60c9b0cf3a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:32.178578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f627e8a-b406-4f22-bba1-60c9b0cf3a33 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.179560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f627e8a-b406-4f22-bba1-60c9b0cf3a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.179909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f627e8a-b406-4f22-bba1-60c9b0cf3a33 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.180763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f627e8a-b406-4f22-bba1-60c9b0cf3a33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:32.181141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f627e8a-b406-4f22-bba1-60c9b0cf3a33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:32.182632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f627e8a-b406-4f22-bba1-60c9b0cf3a33 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:32.198789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 373/738] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:32.200048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f085cf-2f86-4cec-87bf-84b8f7ae717d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.200499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f085cf-2f86-4cec-87bf-84b8f7ae717d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.201216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f085cf-2f86-4cec-87bf-84b8f7ae717d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.205016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0f66d-2522-49b4-9078-af8caf5ea2f4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:32.222862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d5f085cf-2f86-4cec-87bf-84b8f7ae717d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:32.224135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d5f085cf-2f86-4cec-87bf-84b8f7ae717d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:32.255324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:32.255652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.255892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.256220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.271907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:32.274731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:32.306536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:32.307501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:32.308419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:32.308419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:32.308419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:32.308419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:32.308691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 374/739] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 17:30:32.308691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f085cf-2f86-4cec-87bf-84b8f7ae717d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.310958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 366/740] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:32.324537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.418438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:32.431767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:32.443690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f085cf-2f86-4cec-87bf-84b8f7ae717d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:32.461072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f916e-8538-4705-a840-cf27b2091772 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:32.491939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:32.493288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.493568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.493843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.502070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:32.503512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:32.507529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:32.508503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.509000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.509496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.510720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:32.511254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:32.513048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:32.520519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:32.521485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:32.523018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:32.526534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 375/741] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:32.527648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.535773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 367/742] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:32.537207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.659573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5fb84-be57-4d5a-bce6-ebd1519f653b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:32.680043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9c9db-94ff-4de1-82ae-246b8b679976 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:32.708378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:32.708378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.708378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.708843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.709534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:32.712074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:32.716887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:32.724703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5c79-c5fa-472e-9a33-22b352df812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:32.725377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5c79-c5fa-472e-9a33-22b352df812e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.726194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5c79-c5fa-472e-9a33-22b352df812e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.726526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5c79-c5fa-472e-9a33-22b352df812e None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.727374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5c79-c5fa-472e-9a33-22b352df812e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:32.727683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5c79-c5fa-472e-9a33-22b352df812e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:32.729172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5c79-c5fa-472e-9a33-22b352df812e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:32.731031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:32.747404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 368/743] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jul 20 17:30:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:32.754029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e073b-4ee0-47c4-8e85-eda66d8f01fd None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.754320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e073b-4ee0-47c4-8e85-eda66d8f01fd None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.754601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e073b-4ee0-47c4-8e85-eda66d8f01fd None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.755930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 369/744] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:32.756648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e073b-4ee0-47c4-8e85-eda66d8f01fd None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.820463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 376/745] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:30:32 2022] POST /identity/v3/services => generated 240 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:32.820463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:32.911929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e073b-4ee0-47c4-8e85-eda66d8f01fd None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:32.959944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31503cb-a86d-4f09-822a-f975d0dea531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:32.961623 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31503cb-a86d-4f09-822a-f975d0dea531 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:32.962204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31503cb-a86d-4f09-822a-f975d0dea531 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:32.962642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31503cb-a86d-4f09-822a-f975d0dea531 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:32.971268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31503cb-a86d-4f09-822a-f975d0dea531 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:32.973251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31503cb-a86d-4f09-822a-f975d0dea531 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e31503cb-a86d-4f09-822a-f975d0dea531 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:32.997686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:32.998862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:33.000003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:33.002692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 370/746] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:33.004313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b9f9-7feb-46e0-af7f-a33fcf327e06 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.005197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b9f9-7feb-46e0-af7f-a33fcf327e06 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.009941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b9f9-7feb-46e0-af7f-a33fcf327e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.027246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-7b45b9f9-7feb-46e0-af7f-a33fcf327e06 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:33.028810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7b45b9f9-7feb-46e0-af7f-a33fcf327e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:33.034463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd3527-d2dc-4d17-8636-7ca769a6ecc3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:33.085447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:33.086102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.086382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.087267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.092551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:33.134868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 371/747] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:33.136063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45b9f9-7feb-46e0-af7f-a33fcf327e06 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:33.170615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:33.177494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:33.182409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:33.266969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45b9f9-7feb-46e0-af7f-a33fcf327e06 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:33.292669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 377/748] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:30:33.293679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:33.308067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:33.309113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.309743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.310225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.311284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:33.311799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:33.313595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:33.329288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 372/749] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:33.330550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:33.380125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:33.425528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:33.433213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176626a6-a95c-4784-9919-08615a076c6c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:33.460646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f176c-eb05-4358-8e0c-07fcefee27aa None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:33.494312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:33.495122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.495407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.495679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.506596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:33.508037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.508711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.509199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.522271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:33.523119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:33.524843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:33.530756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:33.531727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:33.532888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:33.535511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 378/750] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:33.537917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:33.549415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:33.550333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:33.551248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:33.551248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:33.551248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:33.551248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:33.554850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 373/751] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:33.556691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:33.680757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:33.698876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0e1f-6646-45c7-ac7c-00bc95856358 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:33.733640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835062-d434-4206-a664-a2db12b2d655 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:33.749220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:33.752116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.752592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.753049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.755820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:33.756288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:33.760567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:33.779909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 374/752] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:33.781075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:33.803730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:33.805374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.805917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.806399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.807452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=e3d36cdf82db411e9de1d2f091c16e38) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:33.808165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:33.811737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:33.835833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 379/753] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:33 2022] GET /identity/v3/role_assignments?user.id=e3d36cdf82db411e9de1d2f091c16e38&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:33.837165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:33.903258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:33.910345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422e243-09dc-4648-94e8-a5af6168b9b8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:33.968565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:33.970877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:33.971317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:33.971761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:33.973561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:33.976872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:33.978503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:34.000241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 375/754] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:34.008301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:34.047746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:34.090630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960368-3935-4fa9-8df7-3e7afdce2446 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:34.185851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:34.187456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.187456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.187456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.205588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:34.208827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:34.240867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:34.242337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:34.243243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:34.243243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:34.243243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:34.243243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:34.248157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79d6e3-112e-403f-a692-020b3c5c6566 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:34.255927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 380/755] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:34.256921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:34.317101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80d419-4fa2-4828-8322-46e1130ec8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:34.317957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d419-4fa2-4828-8322-46e1130ec8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.318247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d419-4fa2-4828-8322-46e1130ec8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.318519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d419-4fa2-4828-8322-46e1130ec8d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.339442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80d419-4fa2-4828-8322-46e1130ec8d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:34.340968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80d419-4fa2-4828-8322-46e1130ec8d6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3c80d419-4fa2-4828-8322-46e1130ec8d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:34.359553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:34.360513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:34.361425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:34.365797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 376/756] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:34.367690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cccb-dba9-4e48-9573-b98f62aab6b6 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.367976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cccb-dba9-4e48-9573-b98f62aab6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.368260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cccb-dba9-4e48-9573-b98f62aab6b6 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.373860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 377/757] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:34.377455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cccb-dba9-4e48-9573-b98f62aab6b6 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:34.396172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:34.438094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a93f41-d0d1-4b01-b1ef-484f1500a2e6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:34.468490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:34.476005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828470d-d13d-4e6a-9f61-6d3dec2109ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:34.476744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828470d-d13d-4e6a-9f61-6d3dec2109ce None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.477028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828470d-d13d-4e6a-9f61-6d3dec2109ce None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.477342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828470d-d13d-4e6a-9f61-6d3dec2109ce None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.478476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828470d-d13d-4e6a-9f61-6d3dec2109ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:34.478476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828470d-d13d-4e6a-9f61-6d3dec2109ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:34.479725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828470d-d13d-4e6a-9f61-6d3dec2109ce None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:34.492961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 381/758] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:34.494311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8ccfa-45b8-46b4-8ce7-52b4f8f1673a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.494744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8ccfa-45b8-46b4-8ce7-52b4f8f1673a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.495234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8ccfa-45b8-46b4-8ce7-52b4f8f1673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.512456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cccb-dba9-4e48-9573-b98f62aab6b6 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:34.519947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-30e8ccfa-45b8-46b4-8ce7-52b4f8f1673a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:34.523311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-30e8ccfa-45b8-46b4-8ce7-52b4f8f1673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:34.559316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:34.560346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.560873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.561344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.562344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:34.562848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:34.564626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:34.583609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 378/759] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:34.585033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:34.648670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 382/760] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:34.649608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8ccfa-45b8-46b4-8ce7-52b4f8f1673a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:34.704549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e290b9-7716-474f-8473-4a20cdae5c47 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:34.733242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:34.756915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:34.756915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:34.756915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.756915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.757306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.760686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:34.764226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:34.783482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:34.786233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:34.787172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:34.787717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8ccfa-45b8-46b4-8ce7-52b4f8f1673a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:34.789108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 379/761] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:30:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:34.790155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:34.852026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:34.852716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:34.852994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:34.853267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:34.854385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=2335e75507f44afb8d77f9b70a668a94) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:34.854706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:34.856338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:34.880491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 383/762] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:34 2022] GET /identity/v3/role_assignments?user.id=2335e75507f44afb8d77f9b70a668a94&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:34.880491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:34.985565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4cdf6-33e6-4f5d-bb84-6052d05e9922 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:35.076412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:35.077508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.078015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.078439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.079246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:35.080815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:35.095757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4093-3785-436a-86b6-c43619d73484 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:35.124018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:35.125635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:35.126499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:35.129213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 380/763] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Jul 20 17:30:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:35.137677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:35.152915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:35.163833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5b91-e676-471d-916c-f583cf580c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:35.166321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5b91-e676-471d-916c-f583cf580c83 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.167117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5b91-e676-471d-916c-f583cf580c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.167393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5b91-e676-471d-916c-f583cf580c83 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.169524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5b91-e676-471d-916c-f583cf580c83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:35.169638 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5b91-e676-471d-916c-f583cf580c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:35.171074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5b91-e676-471d-916c-f583cf580c83 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:35.208956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 384/764] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:30:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:35.211646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63ee1-d2fc-408d-9bb0-459a5931a074 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.212029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63ee1-d2fc-408d-9bb0-459a5931a074 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.212196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63ee1-d2fc-408d-9bb0-459a5931a074 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.218163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 385/765] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:35.221179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63ee1-d2fc-408d-9bb0-459a5931a074 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:35.276805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:35.360529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cb32e-2841-4212-a2e3-bed57eaa5fe6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:35.374636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63ee1-d2fc-408d-9bb0-459a5931a074 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:35.419220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:35.419220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.419220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.419220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.427869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:35.431049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:35.431842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91239be5-b0b6-4477-bd6c-7f3b7d30440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:35.433385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91239be5-b0b6-4477-bd6c-7f3b7d30440d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.434887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91239be5-b0b6-4477-bd6c-7f3b7d30440d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.435304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91239be5-b0b6-4477-bd6c-7f3b7d30440d None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.436148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91239be5-b0b6-4477-bd6c-7f3b7d30440d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:35.437680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91239be5-b0b6-4477-bd6c-7f3b7d30440d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:35.440017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91239be5-b0b6-4477-bd6c-7f3b7d30440d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:35.451008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 381/766] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jul 20 17:30:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:35.453330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf2774-c07b-40b8-a5e4-77acde4adb5f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.454002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf2774-c07b-40b8-a5e4-77acde4adb5f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.455430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf2774-c07b-40b8-a5e4-77acde4adb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:35.464409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:35.465668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:35.466648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:35.466648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:35.466648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:35.466648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:35.472394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 386/767] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:35.473817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:35.490872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-34bf2774-c07b-40b8-a5e4-77acde4adb5f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:35.494702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-34bf2774-c07b-40b8-a5e4-77acde4adb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:35.549449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:35.634867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 382/768] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:35.636114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf2774-c07b-40b8-a5e4-77acde4adb5f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:35.685275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2912f-7728-4a4f-bebc-49cc6943839d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:35.700160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:35.726445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:35.727370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.727821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.728250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.729153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:35.729633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:35.731170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:35.738886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 387/769] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jul 20 17:30:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:35.739902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:35.795529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf2774-c07b-40b8-a5e4-77acde4adb5f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:35.871105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:35.871770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:35.872054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:35.872333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:35.873158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:35.873465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:35.874820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:35.905120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 383/770] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:35.905120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:35.936945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:35.941198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e37bd-70f7-46a9-91cb-700d0406fe91 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.020943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.022296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.024345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.025054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] PATH_INFO: `/v3/services/network` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.027129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.028950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:36.051968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:36.053099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:36.053973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:36.061560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 388/771] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:35 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:36.062712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:36.079641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4fce-443b-4fd2-b1b1-9bde8f778c26 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.094336 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:36.140915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.141584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.141869 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.142145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.142947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] RBAC: Adding query filter params (interface=public, service_id=e1635206c915439e9d3b5190e3b0c2c2, region_id=RegionOne) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:36.143255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.145515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.159115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 384/772] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:30:35 2022] GET /identity/v3/endpoints?service_id=e1635206c915439e9d3b5190e3b0c2c2&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:36.159921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:36.208712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2fbb-7dca-4e3b-ac61-e4269db2071c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.251636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e276-4a78-48be-988b-dae34e861e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.252426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e276-4a78-48be-988b-dae34e861e0b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.252722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e276-4a78-48be-988b-dae34e861e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.252995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e276-4a78-48be-988b-dae34e861e0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.264311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820e276-4a78-48be-988b-dae34e861e0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.265954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820e276-4a78-48be-988b-dae34e861e0b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-8820e276-4a78-48be-988b-dae34e861e0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:36.284406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:36.285624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:36.286785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:36.290405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 389/773] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:36.291844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab52c2a-48a5-4be0-becd-4d8747e7e42a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.292160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab52c2a-48a5-4be0-becd-4d8747e7e42a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.292462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab52c2a-48a5-4be0-becd-4d8747e7e42a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.293628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b268caf-3608-48a4-bfb9-c19493fb88cb None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.297901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 390/774] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:36.297901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab52c2a-48a5-4be0-becd-4d8747e7e42a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:36.330485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:36.335921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.337172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.347422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.347422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.347422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] RBAC: Adding query filter params (name=network) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:36.347422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.347422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.349801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 385/775] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:36.351295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:36.402856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:36.418331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab52c2a-48a5-4be0-becd-4d8747e7e42a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.464363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d89c0-05f1-4683-910f-96031df54a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.466939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d89c0-05f1-4683-910f-96031df54a86 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.467452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d89c0-05f1-4683-910f-96031df54a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.467529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d89c0-05f1-4683-910f-96031df54a86 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.468182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d89c0-05f1-4683-910f-96031df54a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.469770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d89c0-05f1-4683-910f-96031df54a86 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.480672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 391/776] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:30:36 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:36.482121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83385bb-7499-4d6a-bba1-37ef16ea500b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.482403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83385bb-7499-4d6a-bba1-37ef16ea500b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.482676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83385bb-7499-4d6a-bba1-37ef16ea500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.494521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c52a7-19f6-4f5c-9cc8-d3e7d8337b44 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.515039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b83385bb-7499-4d6a-bba1-37ef16ea500b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:36.517574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b83385bb-7499-4d6a-bba1-37ef16ea500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:36.545304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.546377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.546868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.547374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.548403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:36.548978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.552008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.567739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 386/777] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:36.569274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:36.610083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 392/778] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:36.612178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83385bb-7499-4d6a-bba1-37ef16ea500b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:36.648443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:36.692352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:36.697328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5013b0c-ff0b-41ec-be03-0a0016c391c7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.746238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.746923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.747208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.747579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.748202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:36.749399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.750147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.758721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 387/779] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:36.758721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:36.769099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83385bb-7499-4d6a-bba1-37ef16ea500b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.843511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:36.844847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:36.845346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:36.845849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/b6f536917f434773a052aa074d5aa07d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:36.849770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=b6f536917f434773a052aa074d5aa07d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:36.852143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:36.932104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f58f80-ecb7-4d18-962a-c4f2113a59d2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:36.941976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 393/780] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:36 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/b6f536917f434773a052aa074d5aa07d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:36.943284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:37.013345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:37.014201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.014491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.014966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.077167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:37.089398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:37.093514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:37.094516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:37.095431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 388/781] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:37.109530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:37.160764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b019e-b660-4b3c-899d-81926be00387 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:37.177394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:37.247004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682d0be-cf17-4e92-859c-8c806c72f43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:37.247616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682d0be-cf17-4e92-859c-8c806c72f43e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.247902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682d0be-cf17-4e92-859c-8c806c72f43e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.248182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682d0be-cf17-4e92-859c-8c806c72f43e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.249037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682d0be-cf17-4e92-859c-8c806c72f43e None admin] RBAC: Adding query filter params (interface=public, service_id=3027e86aaa5c49dbb602d4b83f619bf2, region_id=RegionOne) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:37.249357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682d0be-cf17-4e92-859c-8c806c72f43e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:37.251051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682d0be-cf17-4e92-859c-8c806c72f43e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:37.265139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 394/782] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:30:36 2022] GET /identity/v3/endpoints?service_id=3027e86aaa5c49dbb602d4b83f619bf2&interface=public®ion_id=RegionOne => generated 194 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:37.274930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d660a-9970-411d-946c-204bddbf27ff None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.275317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d660a-9970-411d-946c-204bddbf27ff None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.278895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d660a-9970-411d-946c-204bddbf27ff None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.287601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 395/783] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:37 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:37.292454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d660a-9970-411d-946c-204bddbf27ff None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:37.312917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6af0e9-091a-4083-bcc8-20d6a2f59af0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:37.372764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9fca5-f2ac-4164-a8de-7dbe1630cb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:37.373463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9fca5-f2ac-4164-a8de-7dbe1630cb51 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.373754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9fca5-f2ac-4164-a8de-7dbe1630cb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.374038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9fca5-f2ac-4164-a8de-7dbe1630cb51 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.374939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9fca5-f2ac-4164-a8de-7dbe1630cb51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:37.376525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9fca5-f2ac-4164-a8de-7dbe1630cb51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:37.376736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9fca5-f2ac-4164-a8de-7dbe1630cb51 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:37.394122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 389/784] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:37.397145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995848c-b8d9-4141-a819-638130431e7c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.397478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995848c-b8d9-4141-a819-638130431e7c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.397766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995848c-b8d9-4141-a819-638130431e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.433149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f995848c-b8d9-4141-a819-638130431e7c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:37.433149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f995848c-b8d9-4141-a819-638130431e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:37.435277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d660a-9970-411d-946c-204bddbf27ff None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:37.505533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:37.506131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.506426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.507242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.507937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:37.510492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:37.532221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 396/785] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:30:37 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:37.534796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:37.572523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:37.587310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 390/786] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:37.629537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995848c-b8d9-4141-a819-638130431e7c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:37.661121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:37.719372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b528f-a032-44f2-8bed-44787eb0047c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:37.798693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:37.798693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.798693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.799000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.814627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:37.817428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:37.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:37.841391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:37.842291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:37.842291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:37.842291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:37.842291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:37.844681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 397/787] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:37.850288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:37.856815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995848c-b8d9-4141-a819-638130431e7c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:37.935149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:37.936841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:37.937131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:37.937416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:37.960736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:37.972967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:37.973956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:37.975413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 391/788] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:37.983504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:37.983637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:38.067354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a08bd-02da-47bb-ba9a-51e30a1478e9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:38.148295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:38.153204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:38.153890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:38.154177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:38.154458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:38.155202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:38.155513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:38.157758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:38.182864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 398/789] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:38.190343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:38.207371 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172df78b-0698-4e6f-84dd-b3db25139cb0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:38.266115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:38.266896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:38.267190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:38.267472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:38.268231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:38.268573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:38.269960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:38.285597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 392/790] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:38.293810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:38.444636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d979a35-24ae-43c6-98cb-ee34c4dd80bc None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:38.447397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5911-2d3e-4bcd-9fa5-55678262d9e7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:38.492542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:38.495105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:38.495399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:38.495678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:38.501550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:38.517511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:38.518827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:38.519502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:38.550430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:38.551256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:38.559361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:38.559361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:38.559539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:38.560633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:38.561542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:38.561542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:38.561542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:38.561542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:38.565466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 393/791] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:38.568453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:38.568567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:38.568955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:38.607445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:38.608482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:38.609464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:38.617770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 399/792] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:30:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:38.618635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:38.778492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e09bf-1f88-4e50-9b4c-4012e0aca687 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:38.829914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:38.830600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:38.830901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:38.831184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:38.833583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:38.834027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:38.835996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:38.864347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 394/793] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:38.867722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab15f1-d90d-4a3f-9809-a19c6b54829c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:38.872459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:38.951707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.002810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4fc2c-babd-495a-932c-e71db527f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.003500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4fc2c-babd-495a-932c-e71db527f501 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.004758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4fc2c-babd-495a-932c-e71db527f501 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.006373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4fc2c-babd-495a-932c-e71db527f501 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.007275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4fc2c-babd-495a-932c-e71db527f501 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:39.007585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4fc2c-babd-495a-932c-e71db527f501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.009080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4fc2c-babd-495a-932c-e71db527f501 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.019605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa90c27-b6e8-45cd-ad62-f32e46ed2e27 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.045846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 400/794] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jul 20 17:30:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:39.045846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea167e5-7cb0-4e83-96bf-851f8d14f509 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.045846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea167e5-7cb0-4e83-96bf-851f8d14f509 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.045846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea167e5-7cb0-4e83-96bf-851f8d14f509 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.048140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 401/795] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:39.049232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea167e5-7cb0-4e83-96bf-851f8d14f509 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.077681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e788f-3e47-4955-827b-00b8bbae45e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.077681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e788f-3e47-4955-827b-00b8bbae45e5 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.077681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e788f-3e47-4955-827b-00b8bbae45e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.078067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e788f-3e47-4955-827b-00b8bbae45e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.090372 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e788f-3e47-4955-827b-00b8bbae45e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.091833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e788f-3e47-4955-827b-00b8bbae45e5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.097417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-083e788f-3e47-4955-827b-00b8bbae45e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:39.118748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:39.119736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:39.120747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:39.127894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 395/796] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:39.131517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f66de-e79b-4f35-a2fb-84f1153ebc0b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.132044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f66de-e79b-4f35-a2fb-84f1153ebc0b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.132517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f66de-e79b-4f35-a2fb-84f1153ebc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.159641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-9d6f66de-e79b-4f35-a2fb-84f1153ebc0b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:39.161208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9d6f66de-e79b-4f35-a2fb-84f1153ebc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:39.181423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea167e5-7cb0-4e83-96bf-851f8d14f509 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.222896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.223922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.224683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.224683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.237349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.239348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:39.262057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:39.262993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:39.263887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:39.263887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:39.263887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:39.263887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:39.270925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 402/797] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:39.271409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.285109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 396/798] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:39.286241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f66de-e79b-4f35-a2fb-84f1153ebc0b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.364060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.382201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.398369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce610448-8967-40fb-aa9c-dc0b9e78e6ae None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.437312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f66de-e79b-4f35-a2fb-84f1153ebc0b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.448808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.449659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.449953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.450235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.451146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:39.451466 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.453175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.473929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 403/799] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:39.474873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.482423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f60e6-443e-4e1d-89c3-d3706c2d1059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.483130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f60e6-443e-4e1d-89c3-d3706c2d1059 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.483417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f60e6-443e-4e1d-89c3-d3706c2d1059 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.483694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f60e6-443e-4e1d-89c3-d3706c2d1059 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.485138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f60e6-443e-4e1d-89c3-d3706c2d1059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:39.485451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f60e6-443e-4e1d-89c3-d3706c2d1059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.487000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f60e6-443e-4e1d-89c3-d3706c2d1059 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.505230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 397/800] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:39.505230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9045-3c2f-4694-8e2f-05ad26df43b1 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.505230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9045-3c2f-4694-8e2f-05ad26df43b1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.505230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9045-3c2f-4694-8e2f-05ad26df43b1 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.505899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 398/801] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:39.506653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9045-3c2f-4694-8e2f-05ad26df43b1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.601538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad795f-1421-400e-8cac-1c697a9cae9c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.634021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.641261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9045-3c2f-4694-8e2f-05ad26df43b1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.647592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.650666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.650666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.650666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.650666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.651326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:39.665707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:39.666901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 17:30:39.667927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:39.670069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 404/802] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:39.670954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.686627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.686941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd647ff-4729-4ea4-a095-61ee5c14c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.687718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd647ff-4729-4ea4-a095-61ee5c14c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.688002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd647ff-4729-4ea4-a095-61ee5c14c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.688626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd647ff-4729-4ea4-a095-61ee5c14c3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.702454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd647ff-4729-4ea4-a095-61ee5c14c3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.705313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd647ff-4729-4ea4-a095-61ee5c14c3c6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3bd647ff-4729-4ea4-a095-61ee5c14c3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:39.724380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:39.725883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:39.726844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:39.726844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:39.726844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:39.726844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:39.732422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 399/803] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:39.733811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79755-a6a0-451c-a1c1-978c97c752a1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.734098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79755-a6a0-451c-a1c1-978c97c752a1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.734379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79755-a6a0-451c-a1c1-978c97c752a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.751672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-fcd79755-a6a0-451c-a1c1-978c97c752a1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:39.753079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fcd79755-a6a0-451c-a1c1-978c97c752a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:39.794399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc486b-601f-4c41-b571-e22cd86af5fd None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.843023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:39.843023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:39.843023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:39.843023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:39.843441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 400/804] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:39.843441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79755-a6a0-451c-a1c1-978c97c752a1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.853271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:39.855519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:39.875488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:39.876492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 405/805] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:39.878078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:39.940876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.954969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:39.963550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79755-a6a0-451c-a1c1-978c97c752a1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:39.994860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc345b9-3152-4774-b62c-5ddfe68c617b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.014328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.015349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.015866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.017852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.018822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:40.019345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.020973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.027342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 401/806] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:40.027843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.040931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.041613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.047573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.047573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.047573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:40.047573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.047573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.063063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 406/807] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:40.063063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.158088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceba04-6b5d-406c-819c-f5414163de78 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.176191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c31be-4302-4165-a641-7b54b1e1aea3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.197350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.198080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.198362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.198999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.208703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:40.209550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:40.215087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.215749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.216037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.216316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.217147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:40.224073 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:40.224200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:40.225107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:40.226531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:40.227247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.227324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 402/808] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:40.227324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.228796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:40.232316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 407/809] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:40.233321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.337533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2ed2c-fd79-4844-b354-3aa8c6c58d54 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.344939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6f448-c0af-48e5-92c1-d717e82cd513 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.373788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861f697-0cd0-4c00-9fd3-44f896accfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.374457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861f697-0cd0-4c00-9fd3-44f896accfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.374749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861f697-0cd0-4c00-9fd3-44f896accfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.375026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861f697-0cd0-4c00-9fd3-44f896accfb5 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.375733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861f697-0cd0-4c00-9fd3-44f896accfb5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:40.376057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861f697-0cd0-4c00-9fd3-44f896accfb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.377461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861f697-0cd0-4c00-9fd3-44f896accfb5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.382716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.385086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 403/810] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Jul 20 17:30:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:40.386132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.386421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.386543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afa3dd-4b6b-499f-8649-e83aa0a249c6 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.386698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.386837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afa3dd-4b6b-499f-8649-e83aa0a249c6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.387119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afa3dd-4b6b-499f-8649-e83aa0a249c6 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.392738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 404/811] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:40.392738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afa3dd-4b6b-499f-8649-e83aa0a249c6 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.397823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.399274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:40.414610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:40.415550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:40.416502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:40.418678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 408/812] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:40.419477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.472246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:40.475936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:40.507939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afa3dd-4b6b-499f-8649-e83aa0a249c6 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.530100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a17c-f559-4b06-8dfe-8de858484fbc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.545876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.546803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.547088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.547367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.548123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:40.548421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.549893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.563497 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 405/813] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:40.564259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.567143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62973b3a-f62a-418b-b60f-7af511f6ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.568030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62973b3a-f62a-418b-b60f-7af511f6ebf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.568309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62973b3a-f62a-418b-b60f-7af511f6ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.568628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62973b3a-f62a-418b-b60f-7af511f6ebf0 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/e3d36cdf82db411e9de1d2f091c16e38/roles/713299dfe855451c9fd4127a8cca33fc` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.584214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62973b3a-f62a-418b-b60f-7af511f6ebf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=e3d36cdf82db411e9de1d2f091c16e38, role_id=713299dfe855451c9fd4127a8cca33fc)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.586312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62973b3a-f62a-418b-b60f-7af511f6ebf0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.649549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 409/814] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:40 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/e3d36cdf82db411e9de1d2f091c16e38/roles/713299dfe855451c9fd4127a8cca33fc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:40.666304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e769482-9615-4a66-b554-80f498c3f09b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:40.676580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067d9b-c527-457b-899a-71f863745421 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.677249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067d9b-c527-457b-899a-71f863745421 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.677437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067d9b-c527-457b-899a-71f863745421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.700416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:40.704972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bd067d9b-c527-457b-899a-71f863745421 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:40.706723 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bd067d9b-c527-457b-899a-71f863745421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:40.707676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:40.708424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:40.708777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:40.709057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:40.710394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:40.712139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:40.797474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 410/815] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:40.797474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd067d9b-c527-457b-899a-71f863745421 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.827856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 406/816] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:30:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:40.832004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:40.833208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:40.946271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd067d9b-c527-457b-899a-71f863745421 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:41.015755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:41.016460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:41.016843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:41.017114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:41.019376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:41.019376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:41.020647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:41.021748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41caa-5fc5-42a3-8201-83a46e5cbe94 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:41.044855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 411/817] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:41.045779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:41.077243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:41.078934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:41.079885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:41.079885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:41.106759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:41.106759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:41.134038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:41.134987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:41.135881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:41.135881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:41.135881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:41.135881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:41.138163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 407/818] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:41.139112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:41.177384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:41.184451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:41.202602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128a00-9d10-4300-9b32-35538b95817f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:41.259882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b1ae9-d10f-4d23-a8d6-aeb1f0a5a03a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:41.261865 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:41.262891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:41.266567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:41.267071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:41.267939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:41.269601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:41.288761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:41.289830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 17:30:41.290683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:41.293591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 412/819] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:41 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:41.294705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:41.309681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:41.310658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:41.310939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:41.311210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/2335e75507f44afb8d77f9b70a668a94/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:41.314762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=2335e75507f44afb8d77f9b70a668a94, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:41.318364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:41.438682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 408/820] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:41 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/2335e75507f44afb8d77f9b70a668a94/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:30:41.438682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:41.438955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c6d4-ef7c-4313-8878-d21172fdd1c1 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:41.536458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:41.536458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:41.536458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:41.536458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:41.536876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:41.536876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:41.537734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:41.564459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 413/821] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:41.566947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:41.595218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:41.655288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:41.667305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0086b9d-c892-4fae-8fde-bc32c9e2776c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:41.744256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:41.744904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:41.745188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:41.745466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:41.746182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] RBAC: Adding query filter params (name=network) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:41.746479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:41.747859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:41.758130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 409/822] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:41.763668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:41.788004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782501-faf0-42f6-ac1f-f9404f30b6e9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:41.901911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:41.902733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:41.903026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:41.903306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:41.929358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:41.930851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:41.984801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:41.985818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:41.986780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 414/823] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:30:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:41.997446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:42.042259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68f91-ebb2-4635-b4b1-103d1c7bf226 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:42.105357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:42.106045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:42.106329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:42.106609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:42.107328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:42.107633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:42.109262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:42.124215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 410/824] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:42.133200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:42.185279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:42.274694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:42.368284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0ad1b-9cae-48cf-950c-4fa8cfe1bbe7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:42.391173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f6aea-45f4-4a45-a8a2-15e45955f3f9 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:42.470633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:42.471331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:42.471592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:42.472076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:42.472954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:42.473273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:42.474721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:42.498637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3522f18-0d5a-41e7-97bf-21c79ff8cf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:42.499306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522f18-0d5a-41e7-97bf-21c79ff8cf74 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:42.499584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522f18-0d5a-41e7-97bf-21c79ff8cf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:42.499856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522f18-0d5a-41e7-97bf-21c79ff8cf74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:42.508697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3522f18-0d5a-41e7-97bf-21c79ff8cf74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:42.510108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3522f18-0d5a-41e7-97bf-21c79ff8cf74 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:42.541265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 415/825] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:30:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 544 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:42.549646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:42.648744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 411/826] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:30:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 516 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:43.015455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bfe0e-2bb9-4b63-93ba-6ad2ba9b18f8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.085984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:43.087121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39940d-6668-4dd6-b824-84318e168b04 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.088157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39940d-6668-4dd6-b824-84318e168b04 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.088708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39940d-6668-4dd6-b824-84318e168b04 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.093410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 412/827] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:30:43.099769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720922-56a6-4896-be87-da2730593684 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.101641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720922-56a6-4896-be87-da2730593684 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.102156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720922-56a6-4896-be87-da2730593684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.124278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b2216-381f-4bb0-a57a-0d85f11627e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:43.125004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b2216-381f-4bb0-a57a-0d85f11627e0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.125286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b2216-381f-4bb0-a57a-0d85f11627e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.126778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b2216-381f-4bb0-a57a-0d85f11627e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.127295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c3720922-56a6-4896-be87-da2730593684 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:43.128802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c3720922-56a6-4896-be87-da2730593684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:43.214752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 413/828] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:43.240924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720922-56a6-4896-be87-da2730593684 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:43.253096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:43.331325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b2216-381f-4bb0-a57a-0d85f11627e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:43.332792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b2216-381f-4bb0-a57a-0d85f11627e0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:43.368566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720922-56a6-4896-be87-da2730593684 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.405864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:43.407711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.408378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.409123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.424740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:43.436747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:43.437690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:30:43.438522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:43.446992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 414/829] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:30:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:43.448621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:43.482460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 416/830] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 933 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:30:43.484069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871fe00-6a34-4b59-97d8-c53c97e5b6f5 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.484363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871fe00-6a34-4b59-97d8-c53c97e5b6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.484665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871fe00-6a34-4b59-97d8-c53c97e5b6f5 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.486078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 417/831] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:43.489697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871fe00-6a34-4b59-97d8-c53c97e5b6f5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:43.514482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:43.524740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:43.557362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c9236-3bf0-4a9d-b269-f16077cfebfd None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.591783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eecb13-5832-436e-a47b-871dd58cc155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:43.592383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eecb13-5832-436e-a47b-871dd58cc155 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.592709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eecb13-5832-436e-a47b-871dd58cc155 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.592916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871fe00-6a34-4b59-97d8-c53c97e5b6f5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.593020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eecb13-5832-436e-a47b-871dd58cc155 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.593738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eecb13-5832-436e-a47b-871dd58cc155 None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:43.594050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eecb13-5832-436e-a47b-871dd58cc155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:43.595422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eecb13-5832-436e-a47b-871dd58cc155 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:43.603918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 415/832] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:30:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:43.605248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c783401-35c1-4e72-8f97-36d8ea688fb3 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.605537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c783401-35c1-4e72-8f97-36d8ea688fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.605818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c783401-35c1-4e72-8f97-36d8ea688fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.621673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4c783401-35c1-4e72-8f97-36d8ea688fb3 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:43.622862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4c783401-35c1-4e72-8f97-36d8ea688fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:43.636829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:43.637628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.637916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.638187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.650052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:43.651505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:43.669900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:43.670871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:43.671768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:43.671768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:43.671768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:43.671768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:43.671768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 418/833] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:43.672059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:43.699330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 416/834] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:43.700171 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c783401-35c1-4e72-8f97-36d8ea688fb3 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:43.759930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:43.767876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:43.783425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13633bad-b70a-4e4e-85e3-1756f2ebc80a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.810022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c783401-35c1-4e72-8f97-36d8ea688fb3 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.822734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:43.823530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.823815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.824091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.834295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:43.835773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:43.846080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:43.846683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:43.846969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:43.847247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:43.847973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:43.848281 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:43.849667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:43.853823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:43.854715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:43.855624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:43.855624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:43.855624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:43.855624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:43.859698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 419/835] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:43.860561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:43.866865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 417/836] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:43.866865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:43.964098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d5894-fe1d-46d1-8d7d-575a516576db None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.968416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34544c-f457-4736-addd-2f38ae5c5d3e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:43.987843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:43.994204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:44.002861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.003225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.003528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.003816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.004021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.004124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.004304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.004683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.004860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:44.005147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.005374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.007053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.007295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.023666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 418/837] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:44.023666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:44.023913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:44.025202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:44.026045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 420/838] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:44.026844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.131168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4e992-f87e-4891-9845-6ce8dddf7152 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.140945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0299e6-90b0-454c-96c6-eb96eea50f73 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.169326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.170122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.170406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.170684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.174940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fe8b8-d093-4091-a7e3-4251bcd3e0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.175745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fe8b8-d093-4091-a7e3-4251bcd3e0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.176026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fe8b8-d093-4091-a7e3-4251bcd3e0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.176298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fe8b8-d093-4091-a7e3-4251bcd3e0f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.177416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.179442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.184390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fe8b8-d093-4091-a7e3-4251bcd3e0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.185839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fe8b8-d093-4091-a7e3-4251bcd3e0f1 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:44.193351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:44.194460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:44.195379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 419/839] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:44.195860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-810fe8b8-d093-4091-a7e3-4251bcd3e0f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:44.200479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:44.201454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 421/840] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:44.202789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6605-84a4-4e8f-b7bd-708337d5bddc None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.203446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6605-84a4-4e8f-b7bd-708337d5bddc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.203446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6605-84a4-4e8f-b7bd-708337d5bddc None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.204884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 422/841] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:44.205582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f6605-84a4-4e8f-b7bd-708337d5bddc None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.244213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:44.260576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:44.299226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be58ed-d85a-4d14-bc74-76be72a1da8d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.308329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f6605-84a4-4e8f-b7bd-708337d5bddc None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.332900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.333560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.333844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.334119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.334850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:44.335164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.336574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.340620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfef84f-1909-4af7-a256-0a99c1284067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.340926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 420/842] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:44.341273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfef84f-1909-4af7-a256-0a99c1284067 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.341555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfef84f-1909-4af7-a256-0a99c1284067 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.341689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.341830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfef84f-1909-4af7-a256-0a99c1284067 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.342610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfef84f-1909-4af7-a256-0a99c1284067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:44.342917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfef84f-1909-4af7-a256-0a99c1284067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.344329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfef84f-1909-4af7-a256-0a99c1284067 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.357063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 423/843] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:44.358282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23214-7f6f-46d4-b226-ac5569b666bc None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.358582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23214-7f6f-46d4-b226-ac5569b666bc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.358861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23214-7f6f-46d4-b226-ac5569b666bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.377251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-01b23214-7f6f-46d4-b226-ac5569b666bc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:44.378439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-01b23214-7f6f-46d4-b226-ac5569b666bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:44.440012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee2f21-47b2-4bee-94c7-3bc807a0fad2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.442262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:44.442440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 424/844] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:44.443246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b23214-7f6f-46d4-b226-ac5569b666bc None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.476718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.477385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.477670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.477960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.478984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:44.479457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.481215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.487585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:44.501503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 421/845] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:44.502439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.550921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b23214-7f6f-46d4-b226-ac5569b666bc None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.589752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.590818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.591170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.591447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.592157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:44.592467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.593842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.598624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 425/846] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:44.599406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.608503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bea1d-27a5-48c1-afbd-fdc1802055aa None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.652968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.653688 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.653971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.654378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.655144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=b6f536917f434773a052aa074d5aa07d) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:44.655463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.657504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.666003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 422/847] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/role_assignments?user.id=b6f536917f434773a052aa074d5aa07d&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:44.667427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.705138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:44.726709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1405e-cb90-4419-9c80-7080b21ae216 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.736159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:44.776692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.776907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.777235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.777523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.793455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:44.801419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:44.802431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:44.803916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 426/848] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:44.809767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:44.826398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763255c-d6c8-4502-b2c3-2bf1910c0fc0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.876877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.877473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.877767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:44.878311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:44.959350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b726a4-c2e2-4a6e-815a-ebaf1bf8c88e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:44.982645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:44.984403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:44.998155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:44.999168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:44.999803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.000275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.001388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:45.002006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.003779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.014550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 427/849] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:45.020673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.023146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.061365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.113095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 423/850] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:44 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:30:45.119790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.161883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83451f07-af23-4491-9260-6b56383bb159 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.202766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.203569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.203853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.204128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.215567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.217067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:45.238632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:45.239589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:45.240502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:45.240502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:45.240502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:45.240502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:45.240502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 428/851] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:45.245815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.248554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be603-f30d-4f76-8f38-66638fa8ce14 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.286358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.297325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.298476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.298970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.299665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.311649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.313547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.336684 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:45.337955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:45.338850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:45.339729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:45.339729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:45.339729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:45.339729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:45.344145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 424/852] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:45.346713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.362600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4327a15-197d-49c3-8ee8-a52883eb7650 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.403422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.404946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.405692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.406175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.407627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:45.408673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.410800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.427377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 429/853] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:45.436473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.471367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bbcbc-a3a7-4e17-bf16-7ec68b55512b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.507981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.508847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.509133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.509409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.510248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:45.510605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.512068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.530166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 425/854] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:45.532567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.564905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895e271-ddec-4f91-b8fd-063961f0785e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.575712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.609406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.610555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.611078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.611553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.622766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.624222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.627428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:45.640023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:45.641652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 17:30:45.643018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:45.644761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 430/855] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:45.647121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.648848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e9e19-9835-4cb1-a49b-fba87e812db3 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.697406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.698694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.699194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.699607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.711718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.714045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:45.734519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:45.735461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:45.736352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:45.744543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 426/856] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:45.746248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.773698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed2b4-065e-4feb-8ce7-6e3a3aa836ec None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.818739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.819844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.820303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.820889 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.822712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:45.823208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.824864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.848739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 431/857] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:45.849733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.854963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.859377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfcbff-64df-45fb-9629-27b2ce4e64a3 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:45.901217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:45.902491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:45.903042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:45.903605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:45.904876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:45.905292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:45.906796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:45.924890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 427/858] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:45.925896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:45.926276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:45.981744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994b34b-bcc5-4f45-b49d-a0ee3a1416a6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:46.019323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:46.020078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.020450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.020818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.043712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961303e3-00ef-46ab-9e01-fa3612817ae1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:46.091566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:46.092825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.093321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.093791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.094788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=e3d36cdf82db411e9de1d2f091c16e38) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:46.095256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:46.097510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:46.099271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:46.101559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:46.107602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 428/859] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/role_assignments?user.id=e3d36cdf82db411e9de1d2f091c16e38&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:46.176965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:46.224163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 432/860] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:30:46.230231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:46.261648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:46.337146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fcb6e-4cbf-4b9f-97e8-8e4dd3f1dba9 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:46.371629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:46.372423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.372728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.373002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.383996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:46.385468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:46.402922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:46.403926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:46.404851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:46.404851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:46.404851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:46.404851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:46.404851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 429/861] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:46.411841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:46.521971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db9800-b198-469b-8c89-1b63a9a1a1cf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:46.558827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fb1ab-16fa-4498-8cce-b4cdb8534251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:46.559501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb1ab-16fa-4498-8cce-b4cdb8534251 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.559785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb1ab-16fa-4498-8cce-b4cdb8534251 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.560059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb1ab-16fa-4498-8cce-b4cdb8534251 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.560813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fb1ab-16fa-4498-8cce-b4cdb8534251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:46.561114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fb1ab-16fa-4498-8cce-b4cdb8534251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:46.562617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fb1ab-16fa-4498-8cce-b4cdb8534251 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:46.574834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 433/862] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:46.580306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:46.660922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:46.687877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1a7ca-7b6f-45ce-a4fa-cec6c6b5266a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:46.722849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:46.724007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.724561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.725568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.732626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:46.734459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:46.739866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b099bf4-654d-48aa-adf7-e248282ed1ec None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.740159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b099bf4-654d-48aa-adf7-e248282ed1ec None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.740479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b099bf4-654d-48aa-adf7-e248282ed1ec None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.745665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 434/863] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:46.747356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058768ee-b0c4-4596-a75f-1404d191d767 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.747642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058768ee-b0c4-4596-a75f-1404d191d767 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.747916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058768ee-b0c4-4596-a75f-1404d191d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:46.749928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:46.750801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:46.751689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 430/864] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:46.759481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:46.774481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-058768ee-b0c4-4596-a75f-1404d191d767 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:46.776260 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-058768ee-b0c4-4596-a75f-1404d191d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:46.794918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:46.863430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 435/865] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:46.872455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70de2d-993e-4688-b2a7-48b4dd856b39 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:46.886003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058768ee-b0c4-4596-a75f-1404d191d767 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:46.907388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:46.908153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:46.908457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:46.908748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:46.909596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:46.909953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:46.911614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:46.913392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:46.925584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 431/866] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:46.932143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:46.997538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058768ee-b0c4-4596-a75f-1404d191d767 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:47.043396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:47.044862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:47.045544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:47.046003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:47.046892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:47.048633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:47.057351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f8a42-79e1-4b24-bb31-f20f865a32e0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:47.095977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c1de2-c1c8-449a-8bd4-ec2ef9575598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:47.095977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1de2-c1c8-449a-8bd4-ec2ef9575598 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:47.095977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1de2-c1c8-449a-8bd4-ec2ef9575598 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:47.096332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c1de2-c1c8-449a-8bd4-ec2ef9575598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:47.097152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c1de2-c1c8-449a-8bd4-ec2ef9575598 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=2335e75507f44afb8d77f9b70a668a94) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:47.097470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c1de2-c1c8-449a-8bd4-ec2ef9575598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:47.104077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c1de2-c1c8-449a-8bd4-ec2ef9575598 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:47.105974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 432/867] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:46 2022] GET /identity/v3/role_assignments?user.id=2335e75507f44afb8d77f9b70a668a94&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:47.107376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1874-6b08-4648-b0ff-7a2e455cd96d None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:47.107662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1874-6b08-4648-b0ff-7a2e455cd96d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:47.107935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1874-6b08-4648-b0ff-7a2e455cd96d None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:47.112485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 433/868] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:47.120124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef27b3-d175-40f9-9408-d40728bb1e53 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:47.121521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef27b3-d175-40f9-9408-d40728bb1e53 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:47.122192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef27b3-d175-40f9-9408-d40728bb1e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:47.135171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 436/869] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:30:46 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:30:47.145821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-90ef27b3-d175-40f9-9408-d40728bb1e53 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:47.147699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-90ef27b3-d175-40f9-9408-d40728bb1e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:47.275636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:47.315514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 434/870] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:47.342341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:47.355001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:47.483929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1debca-9296-40df-80c7-9a8346e65b66 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:47.545453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:47.546285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:47.546571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:47.546847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:47.549013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:47.550406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:47.571749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:47.572733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:47.573581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:47.577137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 437/871] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Jul 20 17:30:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:47.586298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef27b3-d175-40f9-9408-d40728bb1e53 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:47.690373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef27b3-d175-40f9-9408-d40728bb1e53 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:47.726966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed544b-ddd1-4d36-8715-3e829d86c7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:47.727629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed544b-ddd1-4d36-8715-3e829d86c7aa None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:47.727910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed544b-ddd1-4d36-8715-3e829d86c7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:47.728184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed544b-ddd1-4d36-8715-3e829d86c7aa None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:47.728912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed544b-ddd1-4d36-8715-3e829d86c7aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:47.729217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed544b-ddd1-4d36-8715-3e829d86c7aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:47.730569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed544b-ddd1-4d36-8715-3e829d86c7aa None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:47.735753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 435/872] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:47.741937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:47.813989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:47.844260 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b7d5e-8a51-4c4a-8433-565495077e7f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:47.879090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eae8e8-1ed0-444f-a1c9-0ba78e37ac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:47.879752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae8e8-1ed0-444f-a1c9-0ba78e37ac2e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:47.880032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae8e8-1ed0-444f-a1c9-0ba78e37ac2e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:47.880307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eae8e8-1ed0-444f-a1c9-0ba78e37ac2e None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:47.881038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae8e8-1ed0-444f-a1c9-0ba78e37ac2e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:47.881944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae8e8-1ed0-444f-a1c9-0ba78e37ac2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:47.883320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eae8e8-1ed0-444f-a1c9-0ba78e37ac2e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:47.892568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 438/873] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:48.381198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139bc58-f12d-4756-91f7-da5e00ffcb92 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:48.382775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139bc58-f12d-4756-91f7-da5e00ffcb92 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:48.383353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139bc58-f12d-4756-91f7-da5e00ffcb92 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:48.385214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 436/874] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:48.390719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1c7e3-2f8c-4793-9443-043db5e58bbd None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:48.391539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1c7e3-2f8c-4793-9443-043db5e58bbd None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:48.392021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1c7e3-2f8c-4793-9443-043db5e58bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:48.409164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-63d1c7e3-2f8c-4793-9443-043db5e58bbd None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:48.410691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-63d1c7e3-2f8c-4793-9443-043db5e58bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:48.480782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 439/875] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:48 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:48.504144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139bc58-f12d-4756-91f7-da5e00ffcb92 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:48.559758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:48.610796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139bc58-f12d-4756-91f7-da5e00ffcb92 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:48.645504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:48.646239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:48.646520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:48.646794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:48.657720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:48.665675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:48.666587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:48.667431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 437/876] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jul 20 17:30:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:48.672619 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1c7e3-2f8c-4793-9443-043db5e58bbd None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:48.747486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:48.776900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1c7e3-2f8c-4793-9443-043db5e58bbd None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:48.817885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:48.818610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:48.818924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:48.819203 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:48.819958 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:48.820266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:48.821670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:48.830266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 440/877] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jul 20 17:30:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:48.837760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:48.943005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218c3f-b8de-40df-b160-6d6b563ca9b8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:48.952032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:48.977307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:48.978105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:48.978389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:48.978664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:48.989450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:48.991026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:49.014549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:49.015529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:49.016416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:49.016416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:49.016416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:49.016416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.016416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 438/878] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:49.023388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.137204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918cb8-8667-4f5e-928f-5aa1ad7cebc2 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:49.169044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:49.177299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7810de-c548-49da-9d78-c0279c87fd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:49.177967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7810de-c548-49da-9d78-c0279c87fd6f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.178838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7810de-c548-49da-9d78-c0279c87fd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.179113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7810de-c548-49da-9d78-c0279c87fd6f None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.179892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7810de-c548-49da-9d78-c0279c87fd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:49.180205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7810de-c548-49da-9d78-c0279c87fd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:49.181768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7810de-c548-49da-9d78-c0279c87fd6f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:49.195096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 441/879] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:49.200587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.304859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb4cd0-6f97-40db-9f86-944f18d74e54 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:49.342383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:49.343293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.343578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.343854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.351650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:49.353115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:49.363859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5280b4-e95b-40e8-a7a8-bf1bb7a4f5a9 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.364420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5280b4-e95b-40e8-a7a8-bf1bb7a4f5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.364891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5280b4-e95b-40e8-a7a8-bf1bb7a4f5a9 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:49.370394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:49.371718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.372997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 439/881] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:49.376152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 442/881] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:49.377494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.381999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17db92-997e-4c96-96e6-6c7c46270473 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.382529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17db92-997e-4c96-96e6-6c7c46270473 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.382875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17db92-997e-4c96-96e6-6c7c46270473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.400031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7d17db92-997e-4c96-96e6-6c7c46270473 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:49.401923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7d17db92-997e-4c96-96e6-6c7c46270473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:49.405792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:49.493755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 443/882] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:49.493755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06a67a-2b34-4072-a7fc-f97d0bf3abcc None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.493755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06a67a-2b34-4072-a7fc-f97d0bf3abcc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.493755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06a67a-2b34-4072-a7fc-f97d0bf3abcc None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.493755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 444/883] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:49.498675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f967c9-5a53-4658-a147-aeaf2693217c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.498790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc60c-1650-419c-9c9e-12d5a904f2ae None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:49.498973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f967c9-5a53-4658-a147-aeaf2693217c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.499235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f967c9-5a53-4658-a147-aeaf2693217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.516702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-72f967c9-5a53-4658-a147-aeaf2693217c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:49.517888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-72f967c9-5a53-4658-a147-aeaf2693217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:49.538492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:49.539157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.539437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.539711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.540570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:49.540876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:49.542308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:49.559893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 440/884] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:49.560923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.589222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 445/885] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:30:49.590149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f967c9-5a53-4658-a147-aeaf2693217c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.649805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:49.655248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:49.678197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0789-b44b-4f75-9a3e-b6f61e1a5452 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:49.708230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f967c9-5a53-4658-a147-aeaf2693217c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:49.714362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:49.715171 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.715457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.715735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.716561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:49.717917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:49.731514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:49.732533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.733378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 441/886] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:49.734740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.749081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:49.750117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.750728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.751097 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.822839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:49.824293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:49.839707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde9051-43de-49c5-9fb5-87543553c211 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:49.873008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:49.873802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:49.874187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:49.874361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:49.875005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:49.876357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:49.878378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:49.888563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:49.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:49.890648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 442/887] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:49.891452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.903039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:49.942075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 446/888] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:30:49.942075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:49.999530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fb4c0-ed7b-4d5e-af8a-24cea6627ac0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.034929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.035586 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.035871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.036146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.036873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:50.037180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.039143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.041568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10511e00-54a4-4693-b7c5-d60232e3071b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.043591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 443/889] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:50.044405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.075616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c575-b198-437e-b347-89497e21b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.076274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c575-b198-437e-b347-89497e21b58b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.076592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c575-b198-437e-b347-89497e21b58b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.076871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c575-b198-437e-b347-89497e21b58b None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.077568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536c575-b198-437e-b347-89497e21b58b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:50.077874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536c575-b198-437e-b347-89497e21b58b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.079229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536c575-b198-437e-b347-89497e21b58b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.083646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 447/890] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:30:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:50.084881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d9e23-394e-434a-95f5-04ad043f2ca7 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.085168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d9e23-394e-434a-95f5-04ad043f2ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.085445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d9e23-394e-434a-95f5-04ad043f2ca7 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.089034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 448/891] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:50.089034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d9e23-394e-434a-95f5-04ad043f2ca7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.101631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.119121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.149431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85109c0f-94e2-4dfe-af05-238002acff3c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.183289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.184080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.184367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.184676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.188783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d9e23-394e-434a-95f5-04ad043f2ca7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.195429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.196898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:50.213484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:50.214527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:50.215622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:50.215622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:50.215622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:50.215622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:50.215622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 444/892] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:50.215622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.223638 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3c778-60df-4a87-ad6b-cc0ffac9259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.224292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3c778-60df-4a87-ad6b-cc0ffac9259f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.224607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3c778-60df-4a87-ad6b-cc0ffac9259f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.224881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3c778-60df-4a87-ad6b-cc0ffac9259f None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.226138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3c778-60df-4a87-ad6b-cc0ffac9259f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:50.226448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3c778-60df-4a87-ad6b-cc0ffac9259f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.227798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3c778-60df-4a87-ad6b-cc0ffac9259f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.232566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 449/893] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:50.233737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835405a9-b99e-4bff-a6ba-fdae0e02da73 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.234026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835405a9-b99e-4bff-a6ba-fdae0e02da73 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.234304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835405a9-b99e-4bff-a6ba-fdae0e02da73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.250307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-835405a9-b99e-4bff-a6ba-fdae0e02da73 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:50.251436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-835405a9-b99e-4bff-a6ba-fdae0e02da73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:50.316389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 450/894] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:50.317239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835405a9-b99e-4bff-a6ba-fdae0e02da73 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.320411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a428c-64c7-438e-9ffa-ea325c34415e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.330321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.343292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.354903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.355566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.355851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.356129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.356860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:50.357169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.358592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.362872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 445/895] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:50.363632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.445942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835405a9-b99e-4bff-a6ba-fdae0e02da73 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.478160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef1ad1-1178-4f66-879c-35ec5a3cd779 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.487498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.488568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.488859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.489139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.490220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:50.490704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.492303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.510008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 451/896] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:50.510008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.520751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.522721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.523348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.523770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.524743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] RBAC: Adding query filter params (interface=public, service_id=2c87ff22dde64d94b5acdf3c1d9f89e4, region_id=RegionOne) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:50.525445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.527571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.534637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 446/897] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/endpoints?service_id=2c87ff22dde64d94b5acdf3c1d9f89e4&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:50.535663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.580063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.589280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.616291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f670dfb-cb78-47a4-a31e-6f3f43b337c6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.639881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abbd2e-b1e7-4eb1-ad09-00f2a7e266fb None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.651511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.652180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.652491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.652777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.653501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.654874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.676313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.677142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.677427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.677702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.685408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.687218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:50.701766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:50.707744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 447/898] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:50.708852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.742836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 452/899] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:30:50 2022] POST /identity/v3/services => generated 269 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:50.744118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:50.835529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b9f4a-b959-4511-b5fc-b754e09e6368 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.850611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.875796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ab3ab-5e9f-472f-b111-670407fcca46 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:50.880977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.882199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.882484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.882764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.883455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.884896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.893892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 448/900] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:30:50.910667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:50.925324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9921e-4fca-47e8-81b0-a820e76bd47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:50.926267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9921e-4fca-47e8-81b0-a820e76bd47a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:50.926873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9921e-4fca-47e8-81b0-a820e76bd47a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:50.927158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9921e-4fca-47e8-81b0-a820e76bd47a None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:50.927972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9921e-4fca-47e8-81b0-a820e76bd47a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:50.928324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9921e-4fca-47e8-81b0-a820e76bd47a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:50.929850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9921e-4fca-47e8-81b0-a820e76bd47a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:50.944772 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 453/901] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:50.951478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:51.066181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037b05d-ed9e-48e1-b75e-0fd678dc9a40 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:51.109359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40e099-42ff-453c-b51b-c4f197c01517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:51.110032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40e099-42ff-453c-b51b-c4f197c01517 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:51.110316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40e099-42ff-453c-b51b-c4f197c01517 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:51.110655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40e099-42ff-453c-b51b-c4f197c01517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:51.111548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40e099-42ff-453c-b51b-c4f197c01517 None admin] RBAC: Adding query filter params (role.id=314d3844043c428e85fb59eedf646924, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=e3d36cdf82db411e9de1d2f091c16e38) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:51.111867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40e099-42ff-453c-b51b-c4f197c01517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:51.113345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40e099-42ff-453c-b51b-c4f197c01517 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:51.120303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 449/902] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:30:50 2022] GET /identity/v3/role_assignments?user.id=e3d36cdf82db411e9de1d2f091c16e38&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=314d3844043c428e85fb59eedf646924 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:52.440549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423f651-b48c-4315-a0c6-cf751771cde2 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:52.441444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423f651-b48c-4315-a0c6-cf751771cde2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:52.441874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423f651-b48c-4315-a0c6-cf751771cde2 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:52.444154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 454/903] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:52.449630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014aa5e-7983-473c-b829-3a58e85709d4 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:52.450465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014aa5e-7983-473c-b829-3a58e85709d4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:52.450952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014aa5e-7983-473c-b829-3a58e85709d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:52.468598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-9014aa5e-7983-473c-b829-3a58e85709d4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:52.470055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9014aa5e-7983-473c-b829-3a58e85709d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:52.501476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:52.542774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 450/904] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:52.565532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423f651-b48c-4315-a0c6-cf751771cde2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:52.675919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423f651-b48c-4315-a0c6-cf751771cde2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:52.712221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb35d8-9aba-42e2-a2a5-439b447343b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:52.713247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb35d8-9aba-42e2-a2a5-439b447343b6 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:52.713739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb35d8-9aba-42e2-a2a5-439b447343b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:52.714216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb35d8-9aba-42e2-a2a5-439b447343b6 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:52.715148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb35d8-9aba-42e2-a2a5-439b447343b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:52.716752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb35d8-9aba-42e2-a2a5-439b447343b6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:52.846214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 455/905] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:30:52 2022] POST /identity/v3/services => generated 241 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:52.900948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ce8a-ed1d-4323-85ca-efb0c4531ee8 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:52.901296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ce8a-ed1d-4323-85ca-efb0c4531ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:52.901500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ce8a-ed1d-4323-85ca-efb0c4531ee8 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:52.903241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 451/906] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:52.909416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba42a-2233-4393-b0de-79b6462122da None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:52.909744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba42a-2233-4393-b0de-79b6462122da None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:52.909975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccba42a-2233-4393-b0de-79b6462122da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:52.928521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4ccba42a-2233-4393-b0de-79b6462122da None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:52.929743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4ccba42a-2233-4393-b0de-79b6462122da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:52.970194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:53.014003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 456/907] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:53.020548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92242-830c-476f-91ea-5056b7255601 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.020847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92242-830c-476f-91ea-5056b7255601 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.021206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92242-830c-476f-91ea-5056b7255601 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.023049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 452/908] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:53.028636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd1a2b-59a0-4cdd-a7df-910ec29cea60 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.029211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd1a2b-59a0-4cdd-a7df-910ec29cea60 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.029491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd1a2b-59a0-4cdd-a7df-910ec29cea60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.040062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92242-830c-476f-91ea-5056b7255601 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.048476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-71bd1a2b-59a0-4cdd-a7df-910ec29cea60 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:53.049675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-71bd1a2b-59a0-4cdd-a7df-910ec29cea60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:53.137456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 457/909] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:53.160786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd1a2b-59a0-4cdd-a7df-910ec29cea60 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.176543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92242-830c-476f-91ea-5056b7255601 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.217088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396c327-d7c7-4015-aa08-25e67fb3eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.217905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396c327-d7c7-4015-aa08-25e67fb3eca7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.218194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396c327-d7c7-4015-aa08-25e67fb3eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.218474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396c327-d7c7-4015-aa08-25e67fb3eca7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.219127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396c327-d7c7-4015-aa08-25e67fb3eca7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.220570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396c327-d7c7-4015-aa08-25e67fb3eca7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-5396c327-d7c7-4015-aa08-25e67fb3eca7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:53.233284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:53.234286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:53.235216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 453/910] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:53.236125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfd70e-dc7c-41f1-ab4a-65b7007ff8b2 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.236125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfd70e-dc7c-41f1-ab4a-65b7007ff8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.236260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfd70e-dc7c-41f1-ab4a-65b7007ff8b2 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.237710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 454/911] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:53.240105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfd70e-dc7c-41f1-ab4a-65b7007ff8b2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.274941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:53.277917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:53.281333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd1a2b-59a0-4cdd-a7df-910ec29cea60 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.320856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f1b88-07ea-4b96-8602-028e1b3a4c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.321665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f1b88-07ea-4b96-8602-028e1b3a4c41 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.321953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f1b88-07ea-4b96-8602-028e1b3a4c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.322241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f1b88-07ea-4b96-8602-028e1b3a4c41 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.322894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f1b88-07ea-4b96-8602-028e1b3a4c41 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.324319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f1b88-07ea-4b96-8602-028e1b3a4c41 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-762f1b88-07ea-4b96-8602-028e1b3a4c41 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:53.336138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:53.337444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 17:30:53.338477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:53.342337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 458/912] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:53.343583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b24e63-d477-483b-9242-484dc6338036 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.343875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b24e63-d477-483b-9242-484dc6338036 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.344155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b24e63-d477-483b-9242-484dc6338036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.361215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d1b24e63-d477-483b-9242-484dc6338036 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:53.361485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfd70e-dc7c-41f1-ab4a-65b7007ff8b2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.362395 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d1b24e63-d477-483b-9242-484dc6338036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:53.397538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.398526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.398976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.399411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.400281 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:53.400763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.402305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.408271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 455/913] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:53.409391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.440586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 459/914] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:53.441601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b24e63-d477-483b-9242-484dc6338036 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.513702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:53.519497 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0771f6-6951-404a-b797-97a78e8ab701 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.545597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:53.561898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b24e63-d477-483b-9242-484dc6338036 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.563916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.564790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.565239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.565736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.566641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:53.567095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.568685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.574392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 456/915] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:53.575276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.603707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.604540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.604888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.605223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.605993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:53.606359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.607806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.620407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 460/916] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:53.620407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.683303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e82b0a-403b-4a35-9f72-0a26f5320e54 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.721317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.721551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24ae5-082c-46e8-92b5-0eab39f0c779 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.722344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.722725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.723072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.734081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:53.742987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:53.744119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:30:53.745335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:53.747526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 457/917] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:53.748901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.759458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.760195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.760569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.760906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.761678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:53.762054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.763511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.768659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 461/918] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:53.769349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.778414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:53.792166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:53.856653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d9dac-bf9f-43c4-9f65-1daa67562387 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.872680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd3b56-e834-4315-8f6f-1b0a0378c05e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:53.892080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.892898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.893283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.894140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.895294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] RBAC: Adding query filter params (interface=public, service_id=f9ee785a85a74414b9bbb8a43f0e233e, region_id=RegionOne) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:53.895705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.898144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.905344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 458/919] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/endpoints?service_id=f9ee785a85a74414b9bbb8a43f0e233e&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:53.906251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:53.912063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:53.913412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:53.913789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:53.914156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:53.914998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:53.915400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:53.916900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:53.925433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 462/920] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:53.926310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:54.003866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:54.016980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:54.021088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d599bb-ddaa-4136-bb15-2ead928eb33d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:54.035327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96819-c29e-4485-9b72-3d5dc616f4d9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:54.058948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:54.059619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:54.059902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:54.060174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:54.061086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:54.062484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:54.071385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:54.072169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:54.072535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:54.075640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:54.075640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:54.075640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:54.081398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 463/921] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:30:53 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:30:54.082189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:54.161921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 459/922] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:30:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:54.234829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf7255-75e8-47cb-ac50-704486473885 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:54.281818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:54.282810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:54.283092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:54.283366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:54.295398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:54.296916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:54.299052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:54.316613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:54.318133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:54.319089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:54.319089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:54.319089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:54.319089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:54.319089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 464/923] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:54.324174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:54.433356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f4e24-9043-486f-8212-87eedabc9e50 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:54.468384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:54.469077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:54.469360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:54.469629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:54.470406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:54.470717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:54.472081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:54.481428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:54.484916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 460/924] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:54.491270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:54.595502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228449c6-2e58-4fd1-9687-ab8d7e72ae7b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:54.637927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:54.639114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:54.639620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:54.640096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:54.647771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:54.649526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:54.665765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:54.666793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:54.667757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:54.674910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 465/925] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:30:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:54.676503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:54.778153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e2182-580a-4896-858c-2f6dfc93a769 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:54.813840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e678352-2a70-4f65-bdaf-5155ab1e9b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:54.814849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e678352-2a70-4f65-bdaf-5155ab1e9b34 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:54.815368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e678352-2a70-4f65-bdaf-5155ab1e9b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:54.815855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e678352-2a70-4f65-bdaf-5155ab1e9b34 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:54.817032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e678352-2a70-4f65-bdaf-5155ab1e9b34 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:54.817580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e678352-2a70-4f65-bdaf-5155ab1e9b34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:54.819218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e678352-2a70-4f65-bdaf-5155ab1e9b34 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:54.841503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 461/926] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jul 20 17:30:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:54.843136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:54.900648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:54.946127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4112f9-f8a4-40ca-935a-146a55e97ca5 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:54.980929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:54.981727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:54.982009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:54.982283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:54.993196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:54.994651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:55.012581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:55.013545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:55.014430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:55.014430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:55.014430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:55.014430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:55.016520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 466/927] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:55.020278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac81b3-3517-4be2-8f5a-a74950b245a9 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.020598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac81b3-3517-4be2-8f5a-a74950b245a9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.020878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac81b3-3517-4be2-8f5a-a74950b245a9 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.022899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 462/928] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:55.023994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.029546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dbfb0-626a-43c7-8ec9-0523366890dc None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.029936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dbfb0-626a-43c7-8ec9-0523366890dc None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.030285 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dbfb0-626a-43c7-8ec9-0523366890dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.058463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-cc1dbfb0-626a-43c7-8ec9-0523366890dc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:55.060063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-cc1dbfb0-626a-43c7-8ec9-0523366890dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:55.134027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:55.142141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5ee9-7be9-45c9-8c02-a198e040bd1b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.151025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:55.154879 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 463/929] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:30:55.178247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dbfb0-626a-43c7-8ec9-0523366890dc None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.187845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.188923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.189281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.189888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.190809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:55.191230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.192721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.213024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 467/930] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:55.214125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.311125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dbfb0-626a-43c7-8ec9-0523366890dc None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.328848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84403da6-6869-4840-8e1e-05cc1aac09bf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.357790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.359010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.359558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.359841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.360769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.362800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.367267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.368163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.368561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.368928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.375940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:55.376476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:55.377666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:55.378629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 464/931] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:55.384046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.385547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:55.399799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:55.400898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:55.401851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:55.405750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 468/932] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:55.407361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.443544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:55.453611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:55.503328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bafe23-b913-4396-8e4e-38063663eca8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.538985 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.539777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.540060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75719e-323e-489e-8692-578c8545c90a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.540517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.540517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.541384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:55.541785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.543233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.553001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 465/933] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:55.554899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.595210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.596216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.597228 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.597679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.599066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:55.599538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.602369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.619060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 469/934] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:55.625215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.669093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d613eb-cb3f-466b-b31f-7c7ee7d3df81 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.685875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:55.705033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.706280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.706280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.706451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.707257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:55.707508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.708915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.719386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 466/935] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:55.720402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.750074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:55.770704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81c42d-6ca9-40c0-b849-89fc810a9d43 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.811386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f43b74-2f8c-45f3-b7f2-5a375ed10a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.812671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f43b74-2f8c-45f3-b7f2-5a375ed10a08 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.813177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f43b74-2f8c-45f3-b7f2-5a375ed10a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.813673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f43b74-2f8c-45f3-b7f2-5a375ed10a08 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/e3d36cdf82db411e9de1d2f091c16e38/roles/314d3844043c428e85fb59eedf646924` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.816094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f43b74-2f8c-45f3-b7f2-5a375ed10a08 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=e3d36cdf82db411e9de1d2f091c16e38, role_id=314d3844043c428e85fb59eedf646924)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.817809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f43b74-2f8c-45f3-b7f2-5a375ed10a08 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.828781 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51ff30-0d30-4571-873c-3049692b8961 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:55.864805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:55.866035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:55.866320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:55.866589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:55.867386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] RBAC: Adding query filter params (interface=public, service_id=135c58c3abf14aeeb9c298d8e755d947, region_id=RegionOne) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:55.867696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:55.869078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:55.874626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 467/936] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/endpoints?service_id=135c58c3abf14aeeb9c298d8e755d947&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:55.880035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:55.916649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 470/937] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:30:55 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/e3d36cdf82db411e9de1d2f091c16e38/roles/314d3844043c428e85fb59eedf646924 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:30:55.937353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:56.008211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892e538-0cb1-4cd4-8b9b-18d98e5a586c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:56.047174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8f08f-2a18-4eef-8460-a103bb83ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:56.048256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8f08f-2a18-4eef-8460-a103bb83ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.048786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8f08f-2a18-4eef-8460-a103bb83ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.049268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8f08f-2a18-4eef-8460-a103bb83ffe2 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.050254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8f08f-2a18-4eef-8460-a103bb83ffe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:56.052015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8f08f-2a18-4eef-8460-a103bb83ffe2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:56.060109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 468/938] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:30:55 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:30:56.238077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddc4f-b000-422a-b298-bcff573854ae None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.238390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddc4f-b000-422a-b298-bcff573854ae None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.238574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddc4f-b000-422a-b298-bcff573854ae None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.240211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 471/939] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:56.246012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060cdc-4bc0-40af-afaa-b4c24d38b6d1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.246306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060cdc-4bc0-40af-afaa-b4c24d38b6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.246591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51060cdc-4bc0-40af-afaa-b4c24d38b6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.263882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-51060cdc-4bc0-40af-afaa-b4c24d38b6d1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:56.265094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-51060cdc-4bc0-40af-afaa-b4c24d38b6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:56.336589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 469/940] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:56.359324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddc4f-b000-422a-b298-bcff573854ae None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:56.395173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cdb11-a602-4f7b-9df1-c4b13ee6d6d6 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.395811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cdb11-a602-4f7b-9df1-c4b13ee6d6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.396323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cdb11-a602-4f7b-9df1-c4b13ee6d6d6 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.397957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 470/941] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:30:56.403489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29809-c2bc-4dcc-8a54-bb30afce1d9c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.403779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29809-c2bc-4dcc-8a54-bb30afce1d9c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.404058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29809-c2bc-4dcc-8a54-bb30afce1d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.422238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-76c29809-c2bc-4dcc-8a54-bb30afce1d9c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:56.423433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-76c29809-c2bc-4dcc-8a54-bb30afce1d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:56.456569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:56.466750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddc4f-b000-422a-b298-bcff573854ae None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:56.504664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:56.505489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.505770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.506079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.506736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:56.508100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:56.508581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 471/942] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:56.521526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:56.523026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.523863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 472/943] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Jul 20 17:30:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:56.529208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29809-c2bc-4dcc-8a54-bb30afce1d9c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:56.534057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:56.547502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:56.639890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc48d-7b00-4bbc-b49a-43d0be266878 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:56.644972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29809-c2bc-4dcc-8a54-bb30afce1d9c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:56.675804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:56.676749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.677201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.677639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.678518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:56.680260 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:56.681860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING keystone.server.flask.common [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 17:30:56.684818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:56.686072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.686825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.686979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.688351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:56.688805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:56.691313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:56.697901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 472/944] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:56.703015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:56.736179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.sql.core [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 17:30:56.736179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 17:30:56.736179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [parameters: {'id': '94aac9f47e804db4881e53539c909120', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 17:30:56.736179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80583) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 17:30:56.741595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94aac9f47e804db4881e53539c909120.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94aac9f47e804db4881e53539c909120. Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application conn.query(q) Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 17:30:56.746753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result.read() Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 17:30:56.747590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application next(self.gen) Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application yield resource Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application next(self.gen) Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise self.value Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application session.commit() Jul 20 17:30:56.749933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.session.flush() Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application compat.raise_( Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 17:30:56.750849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 17:30:56.751762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application conn.query(q) Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application result.read() Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 17:30:56.753269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [parameters: {'id': '94aac9f47e804db4881e53539c909120', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:56.755601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:56.758811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94aac9f47e804db4881e53539c909120. Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 473/945] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Jul 20 17:30:56 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:56.762311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:56.827354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7f395-c848-441f-ac3c-f45a8823e1c8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:56.846052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:56.863214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:56.864135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.864425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.864742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.866109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:56.866334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:56.867715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:56.874595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 473/946] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:56.885349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:56.892801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e1691-e66d-4df1-a58b-54dd841de74f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:56.939354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:56.940159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:56.941003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:56.941297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:56.950929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:56.953001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:30:56.973349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:56.974421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:30:56.975921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:56.979902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 474/947] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:56.981595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:57.000950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:57.007050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12686387-6663-4169-b43d-ac20a665a7fb None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:57.042724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f184ab9-92d1-42e7-8c1e-b9057734b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:57.043418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184ab9-92d1-42e7-8c1e-b9057734b1d0 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:57.043706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184ab9-92d1-42e7-8c1e-b9057734b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:57.043977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184ab9-92d1-42e7-8c1e-b9057734b1d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:57.044748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f184ab9-92d1-42e7-8c1e-b9057734b1d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:57.046192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f184ab9-92d1-42e7-8c1e-b9057734b1d0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:57.107700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a0f71-7af7-4240-bbc4-17a7fb5655d4 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:57.134662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:57.143272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d67d5d-8845-4df8-ac69-2643851747bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:57.143934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67d5d-8845-4df8-ac69-2643851747bc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:57.144222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67d5d-8845-4df8-ac69-2643851747bc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:57.144523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67d5d-8845-4df8-ac69-2643851747bc None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:57.145276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d67d5d-8845-4df8-ac69-2643851747bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:57.145581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d67d5d-8845-4df8-ac69-2643851747bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:57.147589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d67d5d-8845-4df8-ac69-2643851747bc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:57.167464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 475/948] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:57.287571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 474/949] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:30:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:58.038916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75f0d1-b15a-4378-92b0-4d67ceae302c None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.039426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75f0d1-b15a-4378-92b0-4d67ceae302c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.039535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75f0d1-b15a-4378-92b0-4d67ceae302c None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.041186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 476/950] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:58.046475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ec1a5-91c9-44e0-83c2-dcc618f7d702 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.046879 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ec1a5-91c9-44e0-83c2-dcc618f7d702 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.047069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ec1a5-91c9-44e0-83c2-dcc618f7d702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.065668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-591ec1a5-91c9-44e0-83c2-dcc618f7d702 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:58.066859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-591ec1a5-91c9-44e0-83c2-dcc618f7d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:58.145690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 475/951] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:30:58.169863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75f0d1-b15a-4378-92b0-4d67ceae302c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.188274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c362-af9b-458c-bad5-792a4d764610 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.188611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c362-af9b-458c-bad5-792a4d764610 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.188886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63c362-af9b-458c-bad5-792a4d764610 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.190261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 476/952] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:30:58.195283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdaafd-dffc-4293-b88e-0ed21c2c3855 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.195575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdaafd-dffc-4293-b88e-0ed21c2c3855 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.195872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdaafd-dffc-4293-b88e-0ed21c2c3855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.216762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5fbdaafd-dffc-4293-b88e-0ed21c2c3855 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:58.218214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5fbdaafd-dffc-4293-b88e-0ed21c2c3855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:58.279330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75f0d1-b15a-4378-92b0-4d67ceae302c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:58.296882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 477/953] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:30:58.315334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdaafd-dffc-4293-b88e-0ed21c2c3855 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.316171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:58.317238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.317775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.318234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.326615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:58.331012 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:30:58.339775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:58.340966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:30:58.342056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:58.344623 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 477/954] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:58.351290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.417542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:58.448670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdaafd-dffc-4293-b88e-0ed21c2c3855 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:58.486615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:58.487434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.487719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.488006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.488747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:58.490071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:58.495812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e313b4-bd38-4ee1-bd69-80ca21ae3dec None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:58.503213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:58.504317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:58.505224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 478/955] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:58.511816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.541531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:58.542205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.542493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.542774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.543533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:58.543847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:58.545437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:58.559195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 478/956] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:58.560632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.616250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:58.657321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef54b6-a4ae-48c0-a779-e40e0ff46a35 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:58.667042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e11975-b0aa-4f91-b947-390eb1bb9d7f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:58.676886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:58.704149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:58.704355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:58.705220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.705353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.705722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.706176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.706612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.706735 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.708248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:58.708742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:58.711545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:58.722078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 479/957] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:58.722734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.725292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:58.726937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:58.746032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:58.746985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:58.747905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:58.747905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:58.747905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:58.747905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:58.752116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 479/958] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:30:58.753061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.842334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a325-5370-4434-a427-6b9d73f0bd5d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:58.877362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:58.878431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.878934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.879434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.880349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:58.880500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ced2c1-9f96-4236-b135-a6399c8cbb11 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:58.881592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:58.883187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:58.892990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:58.893538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 480/959] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:58.894549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.918356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:58.919076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:58.919367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:58.919644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:58.920491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:58.920787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:58.922205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:58.936872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 480/960] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:30:58.942470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:58.965711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:59.002828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1c482-4a74-42a7-974f-1f77f24f91a5 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.041771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589e8fc-0200-43c9-93e4-747305b0211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:59.042928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589e8fc-0200-43c9-93e4-747305b0211e None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.043613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589e8fc-0200-43c9-93e4-747305b0211e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.044107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589e8fc-0200-43c9-93e4-747305b0211e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.045097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589e8fc-0200-43c9-93e4-747305b0211e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:59.046997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589e8fc-0200-43c9-93e4-747305b0211e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:59.049364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede64da7-76ed-4dc5-81ea-ed3b883b441e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.085432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:59.086245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.086660 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.086994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.099332 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:59.099332 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:30:59.115068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:59.116642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 17:30:59.118292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:59.121608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 481/961] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:30:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:59.122734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:59.132506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 481/962] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:30:58 2022] POST /identity/v3/endpoints => generated 349 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:30:59.244460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:59.268463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe85729-39ee-4d7a-8d4b-f2892ac82de0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.294652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027071b9-f978-4498-b7c7-4d16f33845db None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.294965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027071b9-f978-4498-b7c7-4d16f33845db None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.300573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027071b9-f978-4498-b7c7-4d16f33845db None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.300887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 482/963] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:59.302597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3da78-c68d-458b-8624-f2e8b673ed92 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.304203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3da78-c68d-458b-8624-f2e8b673ed92 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.304203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3da78-c68d-458b-8624-f2e8b673ed92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.316901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:59.317586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.317868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.318142 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.319526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:30:59.319843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:59.321339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:59.328826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-02f3da78-c68d-458b-8624-f2e8b673ed92 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:59.330536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-02f3da78-c68d-458b-8624-f2e8b673ed92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:59.341182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 482/964] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Jul 20 17:30:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:30:59.346713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:59.400681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:59.428427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 483/965] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:30:59.429814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488aab6-6409-4a8b-bf48-83011f861706 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.430108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488aab6-6409-4a8b-bf48-83011f861706 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.430391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488aab6-6409-4a8b-bf48-83011f861706 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.431727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 484/966] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:30:59.436937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94309cc-8618-4009-894c-22ca7b9ccb8c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.437225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94309cc-8618-4009-894c-22ca7b9ccb8c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.437509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94309cc-8618-4009-894c-22ca7b9ccb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.455214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b94309cc-8618-4009-894c-22ca7b9ccb8c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:30:59.456462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b94309cc-8618-4009-894c-22ca7b9ccb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:30:59.464284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64ea01-2d1d-4e8a-9709-82e4613a68ac None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.503596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:59.504237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.504664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.504943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.508522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:59.525739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 485/967] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:30:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:30:59.529388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94309cc-8618-4009-894c-22ca7b9ccb8c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:59.580237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:59.582153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:59.639426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94309cc-8618-4009-894c-22ca7b9ccb8c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.657486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:59.674599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:59.675268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.675551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.675825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.676544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:59.678564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:59.679863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING keystone.server.flask.common [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 17:30:59.689232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 483/968] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:30:59 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 17:30:59.690026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:59.718754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.sql.core [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 17:30:59.718754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 17:30:59.718754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [parameters: {'user_id': '6bdab96f3d404ea9a5b583851f59246f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 17:30:59.718754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80584) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 17:30:59.725204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application conn.query(q) Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application result.read() Jul 20 17:30:59.735308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 17:30:59.736325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application next(self.gen) Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application yield resource Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application next(self.gen) Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise self.value Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application session.commit() Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 17:30:59.739331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self.session.flush() Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application compat.raise_( Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 17:30:59.740780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 17:30:59.742184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application conn.query(q) Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application result.read() Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 17:30:59.743100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [parameters: {'user_id': '6bdab96f3d404ea9a5b583851f59246f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:59.743935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 17:30:59.744755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 486/969] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Jul 20 17:30:59 2022] POST /identity/v3/users => generated 160 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:59.745668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:59.805360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e63bb9-7b0a-42b6-8f96-8d3f4b21e7e4 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.841389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:59.842473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.842774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.843052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.843741 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:59.845146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:59.858933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:59.859929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:30:59.860835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 484/970] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:30:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:59.862460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:59.871096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc5e63-b26a-4a3a-9c63-499899f74cb2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.906254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:30:59.907424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:30:59.907837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:30:59.908069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:30:59.919401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:30:59.921292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:30:59.938900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:30:59.939829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:30:59.941177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:30:59.941177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:30:59.941177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:30:59.941177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:30:59.945577 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 487/971] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:30:59.946301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:30:59.963967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:30:59.967788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2351e0-1c9a-4e8b-b1ab-63116c320977 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:30:59.983742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:00.004233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.005056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.005366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.005645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.012699 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.014134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:31:00.028940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:00.030072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 17:31:00.031244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:00.040168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 485/972] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:30:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:00.040168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:00.062539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd75254-0140-4bb6-8206-c757537d6004 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:00.098434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.099193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.100096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.100395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.101147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:00.101454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.102831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.111454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 488/973] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:30:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:00.112257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:00.154013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050213-63e7-4b59-82d5-802dbf894a30 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:00.192147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.193003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.193421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.193707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.194436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:00.194749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.196134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.212192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 486/974] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:00.212192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:00.213264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:00.220172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fa894-8f66-4b39-93c7-717bc70d3469 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:00.231215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:00.258302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.259720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.260257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.260717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.262125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:00.262584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.264176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.308812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 489/975] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Jul 20 17:31:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:00.309919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:00.313361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb32e4-5848-4ef1-91a5-e8f5d246aec8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:00.355115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.355931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.356328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.356645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.357397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:00.357712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.359315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.365805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 487/976] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:31:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:00.446440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233d7d-37d5-4dee-be6c-2a6ebb175d5b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:00.486007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.486825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.487105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.487379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.495698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.497245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:31:00.512266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:00.513289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:00.514214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:00.514643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 490/977] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:00.519919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:00.619874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:00.630608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5de4b-638f-4b01-8e48-f2538593eacd None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:00.666893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ba3fe-eac7-4adb-9b67-dc5c35945d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.667561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ba3fe-eac7-4adb-9b67-dc5c35945d92 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.667850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ba3fe-eac7-4adb-9b67-dc5c35945d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.668129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ba3fe-eac7-4adb-9b67-dc5c35945d92 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.668953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ba3fe-eac7-4adb-9b67-dc5c35945d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:00.669287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ba3fe-eac7-4adb-9b67-dc5c35945d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.670718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ba3fe-eac7-4adb-9b67-dc5c35945d92 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.682963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 488/978] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:00.688692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:00.704159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:00.789700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae121495-1a0b-4e83-96c9-f92328e3c596 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:00.823461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe831bd5-9b29-4bed-941d-301e88cf9979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:00.824121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831bd5-9b29-4bed-941d-301e88cf9979 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:00.824403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831bd5-9b29-4bed-941d-301e88cf9979 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:00.824716 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831bd5-9b29-4bed-941d-301e88cf9979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:00.825554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831bd5-9b29-4bed-941d-301e88cf9979 None admin] RBAC: Adding query filter params (role.id=314d3844043c428e85fb59eedf646924, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=e3d36cdf82db411e9de1d2f091c16e38) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:00.825872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831bd5-9b29-4bed-941d-301e88cf9979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:00.827320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831bd5-9b29-4bed-941d-301e88cf9979 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:00.833519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 491/979] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:00 2022] GET /identity/v3/role_assignments?user.id=e3d36cdf82db411e9de1d2f091c16e38&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=314d3844043c428e85fb59eedf646924 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:01.326264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde68ed-fc8a-4a56-9d75-f7061d53392e None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:01.326773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde68ed-fc8a-4a56-9d75-f7061d53392e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:01.326875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde68ed-fc8a-4a56-9d75-f7061d53392e None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:01.328232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 489/980] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:01.333505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23445b-8754-4016-bf55-8a07f2827031 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:01.333894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23445b-8754-4016-bf55-8a07f2827031 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:01.334107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23445b-8754-4016-bf55-8a07f2827031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:01.351060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-2e23445b-8754-4016-bf55-8a07f2827031 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:01.352517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2e23445b-8754-4016-bf55-8a07f2827031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:01.424783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 492/981] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:01.445096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde68ed-fc8a-4a56-9d75-f7061d53392e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:01.550070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde68ed-fc8a-4a56-9d75-f7061d53392e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:01.584153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:01.584982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:01.585273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:01.585551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:01.586201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:01.587574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:01.600116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:01.601452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:01.602924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 490/982] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:31:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:01.612676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23445b-8754-4016-bf55-8a07f2827031 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:01.619912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:01.707968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23445b-8754-4016-bf55-8a07f2827031 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:01.741257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:01.741912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:01.742196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:01.742475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:01.743174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:01.743481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:01.744855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:01.749669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 493/983] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:31:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:01.755120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:01.764400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:01.854805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853922e1-51c2-4249-ae04-c850e8ca086b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:01.894365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:01.895056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:01.895343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:01.895624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:01.896350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:01.896713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:01.898125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:01.908843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 491/984] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:31:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:01.914002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:01.988401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:02.020320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057711-f6e9-46db-88a3-9435a225feed None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:02.056802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445167a3-34c8-4d52-8b5b-75d047278a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:02.057463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445167a3-34c8-4d52-8b5b-75d047278a23 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:02.057744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445167a3-34c8-4d52-8b5b-75d047278a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:02.058017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445167a3-34c8-4d52-8b5b-75d047278a23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:02.058817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445167a3-34c8-4d52-8b5b-75d047278a23 None admin] RBAC: Adding query filter params (interface=admin, service_id=135c58c3abf14aeeb9c298d8e755d947, region_id=RegionOne) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:02.059132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445167a3-34c8-4d52-8b5b-75d047278a23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:02.060505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445167a3-34c8-4d52-8b5b-75d047278a23 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:02.065954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 494/985] 10.222.0.42 () {56 vars in 1249 bytes} [Wed Jul 20 17:31:01 2022] GET /identity/v3/endpoints?service_id=135c58c3abf14aeeb9c298d8e755d947&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:02.071875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:02.146320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:02.177128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45609392-983c-4d44-ae4c-26c609bf2854 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:02.213273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a6cc7-73ed-4b08-99f9-8782105567de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:02.213943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6cc7-73ed-4b08-99f9-8782105567de None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:02.214224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6cc7-73ed-4b08-99f9-8782105567de None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:02.214498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a6cc7-73ed-4b08-99f9-8782105567de None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:02.215191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a6cc7-73ed-4b08-99f9-8782105567de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:02.216912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a6cc7-73ed-4b08-99f9-8782105567de None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:02.223475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 492/986] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:31:02 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:31:02.473198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6893-3720-46aa-8e31-2e960d589dbe None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:02.473709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6893-3720-46aa-8e31-2e960d589dbe None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:02.473785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6893-3720-46aa-8e31-2e960d589dbe None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:02.475832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 495/987] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:02.480912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da242cb8-b9e0-4bc9-a487-9c81b372ce33 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:02.481287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da242cb8-b9e0-4bc9-a487-9c81b372ce33 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:02.481461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da242cb8-b9e0-4bc9-a487-9c81b372ce33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:02.498654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-da242cb8-b9e0-4bc9-a487-9c81b372ce33 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:02.500150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-da242cb8-b9e0-4bc9-a487-9c81b372ce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:02.572677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 493/988] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:02.595831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6893-3720-46aa-8e31-2e960d589dbe None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:02.697156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6893-3720-46aa-8e31-2e960d589dbe None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:02.731050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74b168-674b-43b5-9640-2b98ae5f487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:02.731636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74b168-674b-43b5-9640-2b98ae5f487f None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:02.731922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74b168-674b-43b5-9640-2b98ae5f487f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:02.732200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74b168-674b-43b5-9640-2b98ae5f487f None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:02.732951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74b168-674b-43b5-9640-2b98ae5f487f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:02.734318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74b168-674b-43b5-9640-2b98ae5f487f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:02.736532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:02.845580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 496/989] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:31:02 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:02.931962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1441-4ec1-4374-b70c-1e32fa34bce0 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:02.932320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1441-4ec1-4374-b70c-1e32fa34bce0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:02.932570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d1441-4ec1-4374-b70c-1e32fa34bce0 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:02.934403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 494/990] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:02.939881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820b15b-edd3-4180-b2a7-5c370f057837 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:02.940214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820b15b-edd3-4180-b2a7-5c370f057837 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:02.940489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820b15b-edd3-4180-b2a7-5c370f057837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:02.957794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3820b15b-edd3-4180-b2a7-5c370f057837 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:02.959068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3820b15b-edd3-4180-b2a7-5c370f057837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:03.042680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 497/991] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:03.064905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d1441-4ec1-4374-b70c-1e32fa34bce0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:03.138526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:03.176511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d1441-4ec1-4374-b70c-1e32fa34bce0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:03.210818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:03.211620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:03.211908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:03.212187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] PATH_INFO: `/v3/services/image` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:03.212865 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:03.214240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:03.226421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:03.227406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:03.228274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 495/992] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Jul 20 17:31:03 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:03.235055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820b15b-edd3-4180-b2a7-5c370f057837 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:03.343495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820b15b-edd3-4180-b2a7-5c370f057837 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:03.366360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:03.379753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0eff7-1816-40f3-a8f9-384f618c3874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:03.380417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0eff7-1816-40f3-a8f9-384f618c3874 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:03.380724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0eff7-1816-40f3-a8f9-384f618c3874 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:03.381018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0eff7-1816-40f3-a8f9-384f618c3874 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:03.381732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0eff7-1816-40f3-a8f9-384f618c3874 None admin] RBAC: Adding query filter params (name=image) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:03.382178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0eff7-1816-40f3-a8f9-384f618c3874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:03.383448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0eff7-1816-40f3-a8f9-384f618c3874 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:03.389276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 498/993] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:03.394391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:03.498377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507e7c9-0dd7-4ca2-8fcd-2a22f4058810 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:03.523880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:03.531858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86552bdf-e100-4047-9c18-f29412c2a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:03.532813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552bdf-e100-4047-9c18-f29412c2a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:03.533095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552bdf-e100-4047-9c18-f29412c2a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:03.533367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552bdf-e100-4047-9c18-f29412c2a9e0 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:03.534062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86552bdf-e100-4047-9c18-f29412c2a9e0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:03.534364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86552bdf-e100-4047-9c18-f29412c2a9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:03.535722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86552bdf-e100-4047-9c18-f29412c2a9e0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:03.548525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 496/994] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:04.308165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b618-5825-40b8-9028-e378a3d8d208 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:04.308695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b618-5825-40b8-9028-e378a3d8d208 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:04.308772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b618-5825-40b8-9028-e378a3d8d208 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:04.310175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 499/995] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:04.315220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba7da0-9242-45fe-8147-bab0c2b22486 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:04.315515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba7da0-9242-45fe-8147-bab0c2b22486 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:04.315799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba7da0-9242-45fe-8147-bab0c2b22486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:04.332239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d2ba7da0-9242-45fe-8147-bab0c2b22486 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:04.333453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d2ba7da0-9242-45fe-8147-bab0c2b22486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:04.399153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 497/996] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:04.422123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983b618-5825-40b8-9028-e378a3d8d208 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:04.527704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983b618-5825-40b8-9028-e378a3d8d208 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:04.562379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:04.563185 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:04.563469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:04.563742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:04.564393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:04.565784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:04.578249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:04.579262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 17:31:04.580118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:04.584851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 500/997] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Jul 20 17:31:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:04.586178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba7da0-9242-45fe-8147-bab0c2b22486 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:04.687536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba7da0-9242-45fe-8147-bab0c2b22486 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:04.709968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:04.721514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:04.722180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:04.722463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:04.722740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:04.730597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:04.730597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:04.730597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:04.730597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 498/998] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:31:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:04.736664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:04.785805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:04.836709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda11b9-0290-4374-8cf2-bc9adffe856a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:04.870463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10c7a5-8f7c-47aa-bc75-a6c44969308a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:04.871130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10c7a5-8f7c-47aa-bc75-a6c44969308a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:04.871417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10c7a5-8f7c-47aa-bc75-a6c44969308a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:04.871697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10c7a5-8f7c-47aa-bc75-a6c44969308a None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:04.872405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10c7a5-8f7c-47aa-bc75-a6c44969308a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:04.872742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10c7a5-8f7c-47aa-bc75-a6c44969308a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:04.874109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10c7a5-8f7c-47aa-bc75-a6c44969308a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:04.883550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 501/999] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Jul 20 17:31:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:04.885248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:04.952104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b17af-6396-43a9-ab6d-b50e34f624e3 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:04.953242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b17af-6396-43a9-ab6d-b50e34f624e3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:04.953749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b17af-6396-43a9-ab6d-b50e34f624e3 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:04.956020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 502/1000] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:04.961339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4de77-7bfe-43b6-81df-a88a2c0ad67c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:04.961644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4de77-7bfe-43b6-81df-a88a2c0ad67c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:04.961925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4de77-7bfe-43b6-81df-a88a2c0ad67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:04.979530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-adb4de77-7bfe-43b6-81df-a88a2c0ad67c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:04.980753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-adb4de77-7bfe-43b6-81df-a88a2c0ad67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:04.988962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881360-a747-48ea-a196-bcd398e3f3b7 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:05.026129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:05.026862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.027159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.027432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.028228 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:05.029649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:05.050294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 503/1001] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:05.072746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4de77-7bfe-43b6-81df-a88a2c0ad67c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:05.116101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:05.123588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 499/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:31:04 2022] POST /identity/v3/endpoints => generated 325 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:05.178087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4de77-7bfe-43b6-81df-a88a2c0ad67c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:05.222274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:05.223012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.223301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.223581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] PATH_INFO: `/v3/services/compute` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.224243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:05.225702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:05.238235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:05.239342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:31:05.240201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:05.245066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 504/1003] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:31:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:05.245802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:05.298014 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:05.353633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70ae46-79e1-48ba-8b6a-ad8a8ec67928 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:05.388636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:05.389543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.389978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.390404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.391287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:05.391752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:05.393286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:05.398570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 500/1004] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:31:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:05.403965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:05.506262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa314f62-72e0-4122-ac10-79d3196d94ba None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:05.507756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:05.541353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b912f7b-cf83-4704-95c0-759420ce171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:05.542023 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b912f7b-cf83-4704-95c0-759420ce171f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.542311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b912f7b-cf83-4704-95c0-759420ce171f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.542589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b912f7b-cf83-4704-95c0-759420ce171f None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.543296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b912f7b-cf83-4704-95c0-759420ce171f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:05.543605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b912f7b-cf83-4704-95c0-759420ce171f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:05.545012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b912f7b-cf83-4704-95c0-759420ce171f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:05.554712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 505/1005] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:31:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:05.556033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:05.599178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae08f-5a42-48e5-9011-255010663081 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.599518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae08f-5a42-48e5-9011-255010663081 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.600313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae08f-5a42-48e5-9011-255010663081 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.601861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 506/1006] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:31:05.606620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411ab52-9e4a-4b56-82bb-f1ebe9fcf08c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.606990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411ab52-9e4a-4b56-82bb-f1ebe9fcf08c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.607185 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411ab52-9e4a-4b56-82bb-f1ebe9fcf08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.626050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e411ab52-9e4a-4b56-82bb-f1ebe9fcf08c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:05.627236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e411ab52-9e4a-4b56-82bb-f1ebe9fcf08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:05.663081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6ebf0-3ff4-4e67-a9c4-a07c4661486b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:05.675104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:05.699501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:05.700469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 507/1007] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:05 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:05.700603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.700603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.703052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.703052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] RBAC: Adding query filter params (interface=public, service_id=8a9f41b8ce53451f983a78bffa333ac8, region_id=RegionOne) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:05.703052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:05.703357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:05.712751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 501/1008] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:31:05 2022] GET /identity/v3/endpoints?service_id=8a9f41b8ce53451f983a78bffa333ac8&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:05.714419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411ab52-9e4a-4b56-82bb-f1ebe9fcf08c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:05.725565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:05.815119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:05.820899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411ab52-9e4a-4b56-82bb-f1ebe9fcf08c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:05.833591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97feee-e9a7-47c0-80b3-69a54fa700e2 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:05.859711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77d84b-2d6b-4154-a0ea-837496c03391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:05.860388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77d84b-2d6b-4154-a0ea-837496c03391 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.860703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77d84b-2d6b-4154-a0ea-837496c03391 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.860985 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77d84b-2d6b-4154-a0ea-837496c03391 None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.861671 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77d84b-2d6b-4154-a0ea-837496c03391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:05.863042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77d84b-2d6b-4154-a0ea-837496c03391 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:05.869579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 508/1009] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:31:05 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:31:05.870154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33634a30-a380-4580-8e56-d58ffb1b4113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:05.870825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33634a30-a380-4580-8e56-d58ffb1b4113 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:05.871111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33634a30-a380-4580-8e56-d58ffb1b4113 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:05.871391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33634a30-a380-4580-8e56-d58ffb1b4113 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:05.872135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33634a30-a380-4580-8e56-d58ffb1b4113 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:05.873566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33634a30-a380-4580-8e56-d58ffb1b4113 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:05.984462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 502/1010] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:31:05 2022] POST /identity/v3/services => generated 242 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:31:07.179737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c574e-f7bd-4495-942a-9daf3859edfe None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:07.180242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c574e-f7bd-4495-942a-9daf3859edfe None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:07.180319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c574e-f7bd-4495-942a-9daf3859edfe None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:07.181820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 509/1011] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:07.186819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1cb46-6c48-4289-9cf4-56302c83b732 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:07.187209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1cb46-6c48-4289-9cf4-56302c83b732 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:07.187367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1cb46-6c48-4289-9cf4-56302c83b732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:07.204535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f4c1cb46-6c48-4289-9cf4-56302c83b732 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:07.205706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f4c1cb46-6c48-4289-9cf4-56302c83b732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:07.234139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:07.280881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 503/1012] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:07 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:07.304509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c574e-f7bd-4495-942a-9daf3859edfe None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:07.405645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c574e-f7bd-4495-942a-9daf3859edfe None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:07.438794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce65745-6216-44b7-a5b8-14a2a338ef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:07.439590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65745-6216-44b7-a5b8-14a2a338ef10 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:07.439882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65745-6216-44b7-a5b8-14a2a338ef10 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:07.440162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65745-6216-44b7-a5b8-14a2a338ef10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:07.441362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce65745-6216-44b7-a5b8-14a2a338ef10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:07.442808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce65745-6216-44b7-a5b8-14a2a338ef10 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:07.445235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 510/1013] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:07.450737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1cb46-6c48-4289-9cf4-56302c83b732 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:07.560075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1cb46-6c48-4289-9cf4-56302c83b732 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:07.594481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed989e68-4017-4d1e-84f9-c7b7da5c759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:07.595145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989e68-4017-4d1e-84f9-c7b7da5c759b None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:07.595429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989e68-4017-4d1e-84f9-c7b7da5c759b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:07.595707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989e68-4017-4d1e-84f9-c7b7da5c759b None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:07.596464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed989e68-4017-4d1e-84f9-c7b7da5c759b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:07.597904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed989e68-4017-4d1e-84f9-c7b7da5c759b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:07.689245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 504/1014] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:31:07 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:07.924793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62903e-83a1-40d5-aab1-b1f6f33b0dfa None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:07.925161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62903e-83a1-40d5-aab1-b1f6f33b0dfa None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:07.925391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62903e-83a1-40d5-aab1-b1f6f33b0dfa None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:07.926949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 511/1015] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:07.932400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee1ecf-11fd-4cb1-9cab-552d794acc1f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:07.932722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee1ecf-11fd-4cb1-9cab-552d794acc1f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:07.932999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee1ecf-11fd-4cb1-9cab-552d794acc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:07.949971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d1ee1ecf-11fd-4cb1-9cab-552d794acc1f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:07.951072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d1ee1ecf-11fd-4cb1-9cab-552d794acc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:07.959330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.018303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 505/1016] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:07 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:08.041221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62903e-83a1-40d5-aab1-b1f6f33b0dfa None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.046736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621eb7-05e0-4062-b70b-17c0a8b3b6d3 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.047352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621eb7-05e0-4062-b70b-17c0a8b3b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.047864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621eb7-05e0-4062-b70b-17c0a8b3b6d3 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.049696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 506/1017] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:31:08.055468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bed4-8703-4bb8-9778-da43bccf08d0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.055755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bed4-8703-4bb8-9778-da43bccf08d0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.056032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bed4-8703-4bb8-9778-da43bccf08d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.071489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.073391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8b94bed4-8703-4bb8-9778-da43bccf08d0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:08.074572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8b94bed4-8703-4bb8-9778-da43bccf08d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:08.151156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 507/1018] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:08 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:08.161803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62903e-83a1-40d5-aab1-b1f6f33b0dfa None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.174866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94bed4-8703-4bb8-9778-da43bccf08d0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.201956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.202670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.203040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.203321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.203979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.205395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:08.217327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:08.218581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:08.219460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 512/1019] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:08.225204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.254762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94bed4-8703-4bb8-9778-da43bccf08d0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.316915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.326040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579ba74-fcd5-4039-bfb8-4bc4acd93598 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.326990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.327776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.328056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.328330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.329018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.330379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:08.342847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:08.344654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:08.345553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:08.349487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 508/1020] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:08.350427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.361183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.361851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.362155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.362408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.364114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:08.364490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.366574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.371118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 513/1021] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:08.376633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.464731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b61f4-ced0-42b6-a0c5-e16dc063ecc3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.487839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf118e9-68fc-4f92-9495-ac17b67b058d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.498885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.499546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.499831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.500102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.500883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] RBAC: Adding query filter params (name=image) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:08.501198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.502558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.504344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.508384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 509/1022] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:08.514529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.526081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.526976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.527261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.527540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.528237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:08.528592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.529962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.540372 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 514/1023] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:08.541266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.568993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.617037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81d735-1da4-4a7c-80f5-3891c08ed11e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.651492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8d03c-fc91-4423-9d4a-2ad488a52ffa None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.653941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.654700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.654989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.655269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.655970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] RBAC: Adding query filter params (type=image) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:08.656287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.657954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.663400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 510/1024] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:08.669093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.688818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.689483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.689773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.690054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.691347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.693049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.731254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.766939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424b98c-2fbe-4169-845f-948e6e187153 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.780480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 515/1025] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:31:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:08.820146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253deeed-a9fd-48db-8db1-c3fdf9d7f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.820882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253deeed-a9fd-48db-8db1-c3fdf9d7f334 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.821222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253deeed-a9fd-48db-8db1-c3fdf9d7f334 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.821505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253deeed-a9fd-48db-8db1-c3fdf9d7f334 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.822314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253deeed-a9fd-48db-8db1-c3fdf9d7f334 None admin] RBAC: Adding query filter params (interface=public, service_id=309c52723e1b4d9ba1cb52b68df524a3, region_id=RegionOne) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:08.822629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253deeed-a9fd-48db-8db1-c3fdf9d7f334 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.824050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253deeed-a9fd-48db-8db1-c3fdf9d7f334 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.830138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 511/1026] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/endpoints?service_id=309c52723e1b4d9ba1cb52b68df524a3&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:08.836476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:08.921731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:08.949052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42399123-d581-4dd5-94a3-5982fead6340 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:08.983419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c555aa-6866-4f82-9246-be7e73abcced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:08.984013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c555aa-6866-4f82-9246-be7e73abcced None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:08.984297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c555aa-6866-4f82-9246-be7e73abcced None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:08.984602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c555aa-6866-4f82-9246-be7e73abcced None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:08.985283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c555aa-6866-4f82-9246-be7e73abcced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:08.986777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c555aa-6866-4f82-9246-be7e73abcced None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:08.995268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 516/1027] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:31:08 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:31:09.795059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4d760-7b4a-4516-89a1-dfdba9116dd6 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:09.795579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4d760-7b4a-4516-89a1-dfdba9116dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:09.795658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4d760-7b4a-4516-89a1-dfdba9116dd6 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:09.797215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 512/1028] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:09.802321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddbc2c-7823-4b84-bba4-6d93e8b6cd08 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:09.803259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddbc2c-7823-4b84-bba4-6d93e8b6cd08 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:09.803455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddbc2c-7823-4b84-bba4-6d93e8b6cd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:09.819834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-9dddbc2c-7823-4b84-bba4-6d93e8b6cd08 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:09.821039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9dddbc2c-7823-4b84-bba4-6d93e8b6cd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:09.893227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 517/1029] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:09.918183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4d760-7b4a-4516-89a1-dfdba9116dd6 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:10.022877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4d760-7b4a-4516-89a1-dfdba9116dd6 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:10.048916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:10.059389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7a403-deae-4efc-94e2-9d714c3ea830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:10.065826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a403-deae-4efc-94e2-9d714c3ea830 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:10.065826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a403-deae-4efc-94e2-9d714c3ea830 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:10.065826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a403-deae-4efc-94e2-9d714c3ea830 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:10.065826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7a403-deae-4efc-94e2-9d714c3ea830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:10.065826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7a403-deae-4efc-94e2-9d714c3ea830 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:10.065826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 513/1030] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:10.070905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddbc2c-7823-4b84-bba4-6d93e8b6cd08 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:10.152353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:10.183611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddbc2c-7823-4b84-bba4-6d93e8b6cd08 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:10.217374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2f0f6-e099-4fa1-9bac-42e147275898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:10.218055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2f0f6-e099-4fa1-9bac-42e147275898 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:10.218342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2f0f6-e099-4fa1-9bac-42e147275898 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:10.218780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2f0f6-e099-4fa1-9bac-42e147275898 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:10.220052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2f0f6-e099-4fa1-9bac-42e147275898 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:10.222698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2f0f6-e099-4fa1-9bac-42e147275898 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:10.332542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 518/1031] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:31:10 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:10.950304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee50d31-5037-4427-99ed-e44d28a97115 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:10.950816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee50d31-5037-4427-99ed-e44d28a97115 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:10.950893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee50d31-5037-4427-99ed-e44d28a97115 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:10.952373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 514/1032] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:10.957351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86732fca-9821-4410-b7f8-809cb9e669be None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:10.957716 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86732fca-9821-4410-b7f8-809cb9e669be None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:10.957918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86732fca-9821-4410-b7f8-809cb9e669be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:10.974425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-86732fca-9821-4410-b7f8-809cb9e669be None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:10.975610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-86732fca-9821-4410-b7f8-809cb9e669be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:11.042791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 519/1033] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:11.066176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434148f-2325-42d8-ab17-c5c974b9298b None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.066475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434148f-2325-42d8-ab17-c5c974b9298b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.066756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434148f-2325-42d8-ab17-c5c974b9298b None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.068716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 515/1034] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:11.069895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86732fca-9821-4410-b7f8-809cb9e669be None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.073529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798e53d-cab2-4bfd-8553-1c258f305b09 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.073890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798e53d-cab2-4bfd-8553-1c258f305b09 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.074231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798e53d-cab2-4bfd-8553-1c258f305b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.096708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e798e53d-cab2-4bfd-8553-1c258f305b09 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:11.098251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e798e53d-cab2-4bfd-8553-1c258f305b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:11.133096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:11.169659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 516/1035] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:11.179254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86732fca-9821-4410-b7f8-809cb9e669be None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.193423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798e53d-cab2-4bfd-8553-1c258f305b09 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.210548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.211298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.211585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.211863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.221964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:11.230628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:11.231563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:11.232422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 520/1036] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:11.237560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.301426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798e53d-cab2-4bfd-8553-1c258f305b09 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.325605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:11.337455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.338310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.338594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.338876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] PATH_INFO: `/v3/services/image` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.339528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.340931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.345620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37447a-831e-4f1e-a03d-2949657daf9c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:11.353495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:11.354488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:11.355337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 517/1037] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:11.360682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.368418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:11.386418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.393220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.393220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.393220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.393220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:11.393220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.393220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.405142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 521/1038] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:11.405984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.478555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f0de-23b6-4d83-b0c2-62a70aa6c95e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.518093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.518761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.519049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.519334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.519437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd3022-8a5b-4ea4-8e88-c0419c57794f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.520041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] RBAC: Adding query filter params (name=image) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:11.520352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.521842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.527354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 518/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:11.533175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.559949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.561028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.561467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.561746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.572325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.573783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.583967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:11.591364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:11.599550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:11.600507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:31:11.600507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:11.600507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:11.600507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:11.600507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 522/1040] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:11.600507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.632179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:11.639317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc312-5825-4fdf-8ba5-5d0dc9ae2b9b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.673934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.674829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.675113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.675393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.676096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] RBAC: Adding query filter params (type=image) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:11.676405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.678302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.683129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 519/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:11.688681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.706625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a70bc-ed88-482b-917a-a299a7d8d26b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.743790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.744386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.744702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.744987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.745834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:11.746262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.747626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.764107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 523/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:11.765509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.790159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa772827-b2b7-4456-93ba-2a578bb09b83 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.805101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:11.825550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.826214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.832331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.832331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.832331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.832331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.860443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:11.870276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb20a0d-b238-469c-9b3e-ddde9e55d6d9 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:11.902724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:11.903542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:11.903827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:11.904106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:11.912304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:11.913772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:31:11.927389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:11.928952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:11.930356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 524/1043] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:11.934939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:11.940785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 520/1044] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:31:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:31:12.056139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc592a-9473-48af-b75e-8a6cc8efd622 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:12.101800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788aa17-92c0-421a-b7f9-b298c9f63c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:12.102489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788aa17-92c0-421a-b7f9-b298c9f63c2d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.102774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788aa17-92c0-421a-b7f9-b298c9f63c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.103051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788aa17-92c0-421a-b7f9-b298c9f63c2d None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.103969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788aa17-92c0-421a-b7f9-b298c9f63c2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:12.104283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788aa17-92c0-421a-b7f9-b298c9f63c2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:12.105760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788aa17-92c0-421a-b7f9-b298c9f63c2d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:12.126367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 525/1045] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:31:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:12.133591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.236878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8956e5-4a7b-4af5-8e41-2d8b1daff41a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:12.270245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:12.270903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.271899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.272178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.311207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:12.339854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:12.341487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:12.413788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4544f7f-469a-4465-895d-55f0f5537144 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.414113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4544f7f-469a-4465-895d-55f0f5537144 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.414356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4544f7f-469a-4465-895d-55f0f5537144 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.415867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 526/1046] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:31:12.420669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b931e9-99cf-4d5d-83e6-8e35c0d3e01c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.420967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b931e9-99cf-4d5d-83e6-8e35c0d3e01c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.421257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b931e9-99cf-4d5d-83e6-8e35c0d3e01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.428508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:12.438920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-14b931e9-99cf-4d5d-83e6-8e35c0d3e01c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:12.440099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-14b931e9-99cf-4d5d-83e6-8e35c0d3e01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:12.446516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 521/1047] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:12.452050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.511701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 527/1048] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:12.532541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:12.534946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b931e9-99cf-4d5d-83e6-8e35c0d3e01c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.565867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45eb33-71e6-4ffc-aea2-55ea28f3f725 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:12.602284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:12.603363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.603643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.603916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.615797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:12.617266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:12.634988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:12.636110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:12.637089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:31:12.637089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:12.637089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:12.637089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:12.637089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 522/1049] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:12.641621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.647831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b931e9-99cf-4d5d-83e6-8e35c0d3e01c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:12.684923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:12.685708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.685989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.686263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.688486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:12.698561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:12.706297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:12.707263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:12.708120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 528/1050] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:12.713982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.749341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbd11a-63b6-47f2-8a0a-4db181596369 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:12.774240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:12.785793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:12.786462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.786751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.787034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.787759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:12.788078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:12.789488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:12.802042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 523/1051] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:12.807606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.824952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd165ed-c7cb-4243-8c91-7216a0336d82 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:12.861081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:12.861860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.862145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.862420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.863163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:12.863594 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:12.865280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:12.874591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 529/1052] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:12.879755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.913410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f424e-5bdb-4165-8bf6-28f5942d627c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:12.939564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:12.949370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:12.950330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:12.950669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:12.950952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:12.957991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:12.959518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:31:12.973884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:12.975417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:12.976338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 524/1053] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:12.981303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:12.988165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743392b8-fc91-4474-8016-22f6bca4af9b None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:13.025358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:13.026010 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:13.026557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:13.027424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:13.027738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] PATH_INFO: `/v3/users/439dfecec9244c11bbad5d8aabe8b740` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:13.042833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] RBAC: Authorizing `identity:get_user(user_id=439dfecec9244c11bbad5d8aabe8b740)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:13.044282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:13.046368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 530/1054] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/users/439dfecec9244c11bbad5d8aabe8b740 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:13.051588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:13.092011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784cd5d-4a5c-4133-ba63-f2a272c5cd9d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:13.133087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:13.133756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:13.134320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:13.134320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:13.135092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:13.135408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:13.136901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:13.149932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 525/1055] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:13.156067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:13.168959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9db0a-d79a-4a6a-b8d7-e3c462f752b4 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:13.183826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:13.203472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:13.204355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:13.204681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:13.204963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:13.268881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e260c-f170-48bd-97a0-7042181f186b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:13.270957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:13.274058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:13.275504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:13.307559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:13.308209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:13.308525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:13.308794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:13.310489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] RBAC: Adding query filter params (role.id=314d3844043c428e85fb59eedf646924, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=b6f536917f434773a052aa074d5aa07d) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:13.310816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:13.312297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:13.328466 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 526/1056] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:13 2022] GET /identity/v3/role_assignments?user.id=b6f536917f434773a052aa074d5aa07d&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=314d3844043c428e85fb59eedf646924 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:13.383159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 531/1057] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 17:31:13.388061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:13.496198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ea81b-5a53-453f-b3de-a0309d896b3d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:13.532679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083a907-7dc9-485d-9eed-949efa87fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:13.533544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a907-7dc9-485d-9eed-949efa87fc3e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:13.533829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a907-7dc9-485d-9eed-949efa87fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:13.534108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a907-7dc9-485d-9eed-949efa87fc3e None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:13.535285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083a907-7dc9-485d-9eed-949efa87fc3e None admin] RBAC: Authorizing `identity:get_project(project_id=0c50520adc4e4465b5d154ee9b201a51)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:13.537260 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083a907-7dc9-485d-9eed-949efa87fc3e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:13.543946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 527/1058] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:13 2022] GET /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:13.545628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:13.561743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:13.647034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f5a38-ab75-476e-8305-3a80e03feaec None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:13.681442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30398dbe-c5e8-4145-ac04-3536ca1c3308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:13.682100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30398dbe-c5e8-4145-ac04-3536ca1c3308 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:13.682389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30398dbe-c5e8-4145-ac04-3536ca1c3308 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:13.682669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30398dbe-c5e8-4145-ac04-3536ca1c3308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:13.683504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30398dbe-c5e8-4145-ac04-3536ca1c3308 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=0c50520adc4e4465b5d154ee9b201a51, user.id=439dfecec9244c11bbad5d8aabe8b740) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:13.683819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30398dbe-c5e8-4145-ac04-3536ca1c3308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:13.685283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30398dbe-c5e8-4145-ac04-3536ca1c3308 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:13.696469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 532/1059] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:13 2022] GET /identity/v3/role_assignments?user.id=439dfecec9244c11bbad5d8aabe8b740&scope.project.id=0c50520adc4e4465b5d154ee9b201a51&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:14.031242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0492eab-d9c7-4988-b5fa-06703701f35f None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:14.031630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0492eab-d9c7-4988-b5fa-06703701f35f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:14.031816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0492eab-d9c7-4988-b5fa-06703701f35f None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:14.033724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 528/1060] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:14.038828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8895cc-a5a5-4ebb-92b1-6979ade00677 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:14.039128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8895cc-a5a5-4ebb-92b1-6979ade00677 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:14.039421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8895cc-a5a5-4ebb-92b1-6979ade00677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:14.057140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0e8895cc-a5a5-4ebb-92b1-6979ade00677 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:14.058338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0e8895cc-a5a5-4ebb-92b1-6979ade00677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:14.140745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 533/1061] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:14.155384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0492eab-d9c7-4988-b5fa-06703701f35f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:14.191356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:14.258366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0492eab-d9c7-4988-b5fa-06703701f35f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:14.292056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:14.292653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:14.292939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:14.293223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:14.371148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:14.372629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:14.404605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:14.480596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 529/1062] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:14.482070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8895cc-a5a5-4ebb-92b1-6979ade00677 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:14.492203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:14.580532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8895cc-a5a5-4ebb-92b1-6979ade00677 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:14.613557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858e0fc-895e-4e33-a56a-233c209ca45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:14.614349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e0fc-895e-4e33-a56a-233c209ca45a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:14.614637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e0fc-895e-4e33-a56a-233c209ca45a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:14.615450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e0fc-895e-4e33-a56a-233c209ca45a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:14.625299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858e0fc-895e-4e33-a56a-233c209ca45a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:14.626749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858e0fc-895e-4e33-a56a-233c209ca45a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-1858e0fc-895e-4e33-a56a-233c209ca45a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:14.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:14.645203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:14.651454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:31:14.651454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:14.651454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:14.651454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:14.651454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 534/1063] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:14.651739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:14.760631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45307-e2cb-4ed3-ad8f-3fd95391c8d5 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:14.797582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dad760-0e3e-4896-8a28-1db0e34a02c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:14.798254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad760-0e3e-4896-8a28-1db0e34a02c6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:14.798540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad760-0e3e-4896-8a28-1db0e34a02c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:14.798818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dad760-0e3e-4896-8a28-1db0e34a02c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:14.799548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dad760-0e3e-4896-8a28-1db0e34a02c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:14.799859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dad760-0e3e-4896-8a28-1db0e34a02c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:14.801259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dad760-0e3e-4896-8a28-1db0e34a02c6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:14.802919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:14.820477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 530/1064] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:15.345710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b56578-ed7b-4574-a524-dbc04a641309 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.346000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b56578-ed7b-4574-a524-dbc04a641309 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.346280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b56578-ed7b-4574-a524-dbc04a641309 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.347719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 535/1065] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:15.353603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079eb02-3d1b-4116-ad82-17b5c164dae3 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.354032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079eb02-3d1b-4116-ad82-17b5c164dae3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.354291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079eb02-3d1b-4116-ad82-17b5c164dae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.370574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1079eb02-3d1b-4116-ad82-17b5c164dae3 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:15.371766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1079eb02-3d1b-4116-ad82-17b5c164dae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:15.441039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 531/1066] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:15.463755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b56578-ed7b-4574-a524-dbc04a641309 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:15.522396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:15.567359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b56578-ed7b-4574-a524-dbc04a641309 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:15.601110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab13589-545a-4f26-902e-5e629a2d42e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:15.601844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13589-545a-4f26-902e-5e629a2d42e6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.602127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13589-545a-4f26-902e-5e629a2d42e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.602410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13589-545a-4f26-902e-5e629a2d42e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.612175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab13589-545a-4f26-902e-5e629a2d42e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-7ab13589-545a-4f26-902e-5e629a2d42e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:15.620834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:15.622030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:15.623228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 536/1067] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jul 20 17:31:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:15.628025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079eb02-3d1b-4116-ad82-17b5c164dae3 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:15.715192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b997df-95f4-4586-a38c-2bbff15d3005 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.715481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b997df-95f4-4586-a38c-2bbff15d3005 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.715758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b997df-95f4-4586-a38c-2bbff15d3005 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.717170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 537/1068] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:31:15.721840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140251c8-5dc9-4ca7-86bc-7263bcbd64f5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.722126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140251c8-5dc9-4ca7-86bc-7263bcbd64f5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.722400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140251c8-5dc9-4ca7-86bc-7263bcbd64f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.726336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079eb02-3d1b-4116-ad82-17b5c164dae3 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:15.739326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-140251c8-5dc9-4ca7-86bc-7263bcbd64f5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:15.741106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-140251c8-5dc9-4ca7-86bc-7263bcbd64f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:15.749936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:15.760180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:15.760874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.761160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.762170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.762170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:15.762471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:15.763856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:15.771815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 532/1069] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jul 20 17:31:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:15.777472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:15.808259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 538/1070] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:15.830892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140251c8-5dc9-4ca7-86bc-7263bcbd64f5 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:15.875086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8e064-710d-40ee-96ea-3bbd6bbe8177 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:15.879182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:15.906894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:15.908029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.908317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.908634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.919526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:15.921010 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:15.928600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140251c8-5dc9-4ca7-86bc-7263bcbd64f5 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:15.938136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:15.939125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:15.940453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:31:15.940453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:15.940453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:15.940453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:15.940453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 533/1071] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:15.945348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:15.966146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:15.966933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:15.967225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:15.967510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:15.977975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:15.984285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:15.985499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:15.986485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:15.987385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 539/1072] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:31:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:15.992614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.043836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a28ba0-0c85-4826-ba11-1037b19bd4c6 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.077552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.078211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.078498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.078774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.079585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:16.079904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.081312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.092095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f82c4-a9b1-40d8-a679-df01a0d1d8b9 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.093003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 534/1073] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:16.098596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.103663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:16.126700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.127294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.127576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.127851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.128679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:16.128983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.130425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.139422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 540/1074] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:31:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:16.145022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.194722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:16.198474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ac7d2-8ecf-41ee-a367-b2d9c6d34342 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.231301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.232090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.232374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.232683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.239658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e94e-c097-49a5-8894-bb5ee4742b25 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.240327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.241981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:31:16.256536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:16.257482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:16.258446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 535/1075] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:16.263346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.277246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.278145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.278437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.278712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] PATH_INFO: `/v3/users/439dfecec9244c11bbad5d8aabe8b740` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.279903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] RBAC: Authorizing `identity:get_user(user_id=439dfecec9244c11bbad5d8aabe8b740)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.281364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.283238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 541/1076] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/users/439dfecec9244c11bbad5d8aabe8b740 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:16.289546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.328697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:16.360251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb21855-4a1f-4ac7-9a32-0fa2de69d7ca None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.385003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cd8c9-33a8-4b1f-87a9-cfbbb90ef50f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.394081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.394745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.395039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.395319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.396742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:16.397058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.398490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.415644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 536/1077] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:16.420343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.420343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.420343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.420343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.420920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] RBAC: Authorizing `identity:get_project(project_id=0c50520adc4e4465b5d154ee9b201a51)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.421245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.422344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.428827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 542/1078] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:16.430301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.438300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:16.517814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce8836-60c3-4ea4-8d40-a1877bf93ae5 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.529398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e3f56-484e-4925-9321-0cb070fb0d15 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.538636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:16.551279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.552065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.552376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.552682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.562196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.562416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.563069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.563352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.563643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51/users/439dfecec9244c11bbad5d8aabe8b740/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.563871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.565609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] RBAC: Authorizing `identity:create_grant(project_id=0c50520adc4e4465b5d154ee9b201a51, user_id=439dfecec9244c11bbad5d8aabe8b740, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.567225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:16.581383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:16.585185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:16.586091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:31:16.586091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:16.586091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:16.586091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:16.586091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 537/1079] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:16.588965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.629885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 543/1080] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:31:16 2022] PUT /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51/users/439dfecec9244c11bbad5d8aabe8b740/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:31:16.655704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:16.691782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762622bc-0f4b-4f5f-a723-1dcf8a876859 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.735554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.736238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.736554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.736839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.737579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:16.737892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.739296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.751766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 538/1081] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:16.758199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:16.859709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0c2ff-831e-4fbf-be58-5355c0bb799a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:16.893330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:16.894124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:16.894411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:16.894687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:16.901218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:16.902668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:31:16.916398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:16.917495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:16.918432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 544/1082] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:16.923350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:17.021154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2daa7d-4399-44fb-b9b2-59d9dcc30ff4 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:17.046606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:17.054869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17002689-4e43-4703-bea7-2a97fc293665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:17.059250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17002689-4e43-4703-bea7-2a97fc293665 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:17.059250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17002689-4e43-4703-bea7-2a97fc293665 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:17.059250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17002689-4e43-4703-bea7-2a97fc293665 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:17.059250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17002689-4e43-4703-bea7-2a97fc293665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:17.059250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17002689-4e43-4703-bea7-2a97fc293665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:17.059250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17002689-4e43-4703-bea7-2a97fc293665 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:17.071281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 539/1083] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:17.077337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:17.110078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:17.180864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419e05-8e3c-4674-92aa-28d1a82f548d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:17.213767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02565a14-8362-479e-bf51-01e50763e8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:17.214653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02565a14-8362-479e-bf51-01e50763e8e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:17.214936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02565a14-8362-479e-bf51-01e50763e8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:17.215212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02565a14-8362-479e-bf51-01e50763e8e5 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/b6f536917f434773a052aa074d5aa07d/roles/314d3844043c428e85fb59eedf646924` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:17.217433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02565a14-8362-479e-bf51-01e50763e8e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=b6f536917f434773a052aa074d5aa07d, role_id=314d3844043c428e85fb59eedf646924)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:17.218856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02565a14-8362-479e-bf51-01e50763e8e5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:17.342252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 545/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:31:17 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/b6f536917f434773a052aa074d5aa07d/roles/314d3844043c428e85fb59eedf646924 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:31:18.650421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c9f6c-1209-4fd1-bae2-cb2078716170 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:18.650949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c9f6c-1209-4fd1-bae2-cb2078716170 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:18.651038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c9f6c-1209-4fd1-bae2-cb2078716170 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:18.652454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 540/1085] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:18.657934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d45038-74fc-4443-838a-ea91102579e0 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:18.658302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d45038-74fc-4443-838a-ea91102579e0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:18.658711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d45038-74fc-4443-838a-ea91102579e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:18.676237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-46d45038-74fc-4443-838a-ea91102579e0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:18.678026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-46d45038-74fc-4443-838a-ea91102579e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:18.743571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 546/1086] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:18.767003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c9f6c-1209-4fd1-bae2-cb2078716170 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:18.866096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c9f6c-1209-4fd1-bae2-cb2078716170 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:18.899443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:18.900238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:18.900615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:18.900843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:18.911023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:18.918672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:18.919747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:18.920792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:18.924752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 541/1087] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:31:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:18.925931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d45038-74fc-4443-838a-ea91102579e0 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:18.944068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:19.022329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d45038-74fc-4443-838a-ea91102579e0 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.055286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.055951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.056233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.056533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.057274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:19.057577 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:19.058935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:19.066711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 547/1088] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:31:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:19.073052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:19.124560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:19.174047 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175948-8a96-4e4a-82f8-c96d17e7c53a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.206319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a0654-f507-4f18-a517-d400a7561b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.207178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a0654-f507-4f18-a517-d400a7561b11 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.207459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a0654-f507-4f18-a517-d400a7561b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.207733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a0654-f507-4f18-a517-d400a7561b11 None admin] PATH_INFO: `/v3/users/439dfecec9244c11bbad5d8aabe8b740` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.209468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a0654-f507-4f18-a517-d400a7561b11 None admin] RBAC: Authorizing `identity:get_user(user_id=439dfecec9244c11bbad5d8aabe8b740)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:19.210891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a0654-f507-4f18-a517-d400a7561b11 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:19.212530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 542/1089] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/users/439dfecec9244c11bbad5d8aabe8b740 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:19.218227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:19.295958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:19.313755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec133b10-c830-40ff-be96-d0143911ad04 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.340099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9896-bbf5-4ca1-9a2c-16c369bd1212 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.340383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9896-bbf5-4ca1-9a2c-16c369bd1212 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.340688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce9896-bbf5-4ca1-9a2c-16c369bd1212 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.342041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 543/1090] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:31:19.346748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135d8af-10ae-41c5-b077-0d6c7901a57f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.346940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.347053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135d8af-10ae-41c5-b077-0d6c7901a57f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.347291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135d8af-10ae-41c5-b077-0d6c7901a57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.347585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.347872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.348153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.365539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e135d8af-10ae-41c5-b077-0d6c7901a57f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:19.366708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e135d8af-10ae-41c5-b077-0d6c7901a57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:19.415494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:19.416964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:19.435283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 544/1091] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:19.458540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135d8af-10ae-41c5-b077-0d6c7901a57f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:19.494013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:19.504349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:19.513213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 548/1092] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:19.519061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:19.556255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135d8af-10ae-41c5-b077-0d6c7901a57f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.593923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.594740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.595025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.595664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.607118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:19.615478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:19.616472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:19.617337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 545/1093] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:31:19.618041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3f79f-8da7-4832-a4f0-c725e5b8a186 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.622686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:19.651001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.651850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.652124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.652392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.653610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] RBAC: Authorizing `identity:get_project(project_id=0c50520adc4e4465b5d154ee9b201a51)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:19.655028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:19.656740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 549/1094] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:19.663459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:19.723413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:19.733650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6003b-5110-49a5-a547-ad87b031c779 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.742045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:19.767722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.768385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.768701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.768977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.769720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:19.770035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:19.771974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:19.773463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb55d59-f5d1-42ec-8219-f4ad639e14cf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.780492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 546/1095] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:19.786286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:19.807929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.808621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.808905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.809180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.810001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=0c50520adc4e4465b5d154ee9b201a51, user.id=439dfecec9244c11bbad5d8aabe8b740) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:19.810308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:19.811728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:19.817581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 550/1096] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/role_assignments?user.id=439dfecec9244c11bbad5d8aabe8b740&scope.project.id=0c50520adc4e4465b5d154ee9b201a51&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:19.888605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51f061-ad03-437d-b26a-a0ed791a6623 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:19.923369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:19.924178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:19.924492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:19.924773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:19.936309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:19.937824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:19.956646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:19.957790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:19.958927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:31:19.958927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:19.958927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:19.958927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:19.958927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 547/1097] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:19.964417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:20.068678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06512d88-168c-4172-a48d-c9197f5f50d6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:20.082360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:20.102533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:20.103202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:20.103486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:20.103765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:20.104517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:20.104829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:20.106196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:20.118530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 551/1098] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:20.124092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:20.145665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:20.223876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ba245-7ca6-4d99-b203-3022d5c206af None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:20.257452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:20.258247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:20.258532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:20.258810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:20.265847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:20.267285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:31:20.281353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:20.282561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:20.283599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 548/1099] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Jul 20 17:31:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:20.289062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:20.393264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560df5a-f2ca-4e52-92fb-4cc2819c3aa0 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:20.425894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:20.426556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:20.426843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:20.427168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:20.428537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:20.428852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:20.430292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:20.447299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 552/1100] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Jul 20 17:31:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:20.453120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:20.532358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:20.550692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3a38c-9ea2-4cd2-9e2c-2d0fa817d7f3 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:20.584724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:20.585366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:20.585649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:20.585920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:20.649778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:20.651155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:20.726057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:20.744490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 549/1101] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:20.750314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:20.780226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:20.848364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfa4c-3b8a-4f7c-827d-536d5815c6e4 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:20.881502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:20.882290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:20.882580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:20.882858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:20.892573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:20.894011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:20.911132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:20.912724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:20.913694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:31:20.913694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:20.913694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:31:20.913694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:20.913694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 553/1102] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:20.918584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:21.014470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0077c-5c80-4adf-ac6f-32b717427f0a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:21.047043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:21.047695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:21.047975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:21.048251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:21.048993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:21.049298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:21.050661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:21.061920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 550/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:21.067500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:21.146733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:21.179121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d4cb7-3ab4-40ec-bfc4-56385a911a28 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:21.196936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:21.197730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:21.198216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:21.198725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:21.205565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:21.206998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:31:21.220596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:21.221603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:21.222542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 554/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:31:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:21.228107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:21.268957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:21.325891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b44f4-61a1-4a09-9f85-e5e798df0dda None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:21.358673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d58945-8b7a-4fac-9268-2b6786b43ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:21.359332 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d58945-8b7a-4fac-9268-2b6786b43ebf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:21.359621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d58945-8b7a-4fac-9268-2b6786b43ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:21.359901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d58945-8b7a-4fac-9268-2b6786b43ebf None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:21.360710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d58945-8b7a-4fac-9268-2b6786b43ebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:21.361025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d58945-8b7a-4fac-9268-2b6786b43ebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:21.362466 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d58945-8b7a-4fac-9268-2b6786b43ebf None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:21.374856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 551/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:31:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:21.380942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:21.482742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaa1fe-da4b-42e0-893f-284d7af67687 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:21.518876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e6928-ef20-4e31-ba4a-4228bd998ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:21.519538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e6928-ef20-4e31-ba4a-4228bd998ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:21.519825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e6928-ef20-4e31-ba4a-4228bd998ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:21.520109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e6928-ef20-4e31-ba4a-4228bd998ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:21.521513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e6928-ef20-4e31-ba4a-4228bd998ef4 None admin] RBAC: Adding query filter params (role.id=314d3844043c428e85fb59eedf646924, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=b6f536917f434773a052aa074d5aa07d) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:21.521832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e6928-ef20-4e31-ba4a-4228bd998ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:21.523259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e6928-ef20-4e31-ba4a-4228bd998ef4 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:21.529188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 555/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:21 2022] GET /identity/v3/role_assignments?user.id=b6f536917f434773a052aa074d5aa07d&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=314d3844043c428e85fb59eedf646924 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:21.885756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a366ce-3588-4507-af86-9c67e6ebdfa2 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:21.886233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a366ce-3588-4507-af86-9c67e6ebdfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:21.886233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a366ce-3588-4507-af86-9c67e6ebdfa2 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:21.887610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 552/1107] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:21.893074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba8818-165d-4568-aa2e-3d7db2fcdec7 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:21.893510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba8818-165d-4568-aa2e-3d7db2fcdec7 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:21.893624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba8818-165d-4568-aa2e-3d7db2fcdec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:21.905227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:21.912139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f5ba8818-165d-4568-aa2e-3d7db2fcdec7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:21.913366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f5ba8818-165d-4568-aa2e-3d7db2fcdec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:21.984239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 556/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:22.009347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a366ce-3588-4507-af86-9c67e6ebdfa2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:22.109373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:22.113165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a366ce-3588-4507-af86-9c67e6ebdfa2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:22.148144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4dc11-9d16-4ed1-8815-7b14a8f6c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:22.148979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4dc11-9d16-4ed1-8815-7b14a8f6c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:22.149273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4dc11-9d16-4ed1-8815-7b14a8f6c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:22.149551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4dc11-9d16-4ed1-8815-7b14a8f6c8e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:22.150737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4dc11-9d16-4ed1-8815-7b14a8f6c8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:22.152192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4dc11-9d16-4ed1-8815-7b14a8f6c8e5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:22.153855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 553/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:22.159873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba8818-165d-4568-aa2e-3d7db2fcdec7 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:22.262814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba8818-165d-4568-aa2e-3d7db2fcdec7 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:22.296698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7776b1-cd41-4a0c-ba75-08adcb44d98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:22.297374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7776b1-cd41-4a0c-ba75-08adcb44d98f None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:22.297660 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7776b1-cd41-4a0c-ba75-08adcb44d98f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:22.297939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7776b1-cd41-4a0c-ba75-08adcb44d98f None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:22.298659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7776b1-cd41-4a0c-ba75-08adcb44d98f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:22.300098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7776b1-cd41-4a0c-ba75-08adcb44d98f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:22.308156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:22.553800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 557/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:31:22 2022] POST /identity/v3/users => generated 286 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:24.603551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440bfc1-4e68-4cba-b35b-f4a44af322ff None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:24.604108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440bfc1-4e68-4cba-b35b-f4a44af322ff None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:24.604108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440bfc1-4e68-4cba-b35b-f4a44af322ff None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:24.605524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 554/1111] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:24.610877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fbb75-4958-488e-8aed-4c036ac6af9c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:24.611309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fbb75-4958-488e-8aed-4c036ac6af9c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:24.611443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fbb75-4958-488e-8aed-4c036ac6af9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:24.628340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0d1fbb75-4958-488e-8aed-4c036ac6af9c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:24.629550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0d1fbb75-4958-488e-8aed-4c036ac6af9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:24.694081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 558/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:24.717072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440bfc1-4e68-4cba-b35b-f4a44af322ff None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:24.814314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440bfc1-4e68-4cba-b35b-f4a44af322ff None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:24.845947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:24.846778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:24.847057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:24.847329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] PATH_INFO: `/v3/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:24.849301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] RBAC: Authorizing `identity:get_role(role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:24.850675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:24.852009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 555/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:24 2022] GET /identity/v3/roles/4182f79f5cda459497c049f3f07a1713 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:24.858160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fbb75-4958-488e-8aed-4c036ac6af9c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:24.955351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fbb75-4958-488e-8aed-4c036ac6af9c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:24.977205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:24.988079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:24.988962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:24.989248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:24.989571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] PATH_INFO: `/v3/users/861586e08f7a4db3b9a44a9ad1f0d1af` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:25.002061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] RBAC: Authorizing `identity:get_user(user_id=861586e08f7a4db3b9a44a9ad1f0d1af)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:25.003491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:25.005661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 559/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:24 2022] GET /identity/v3/users/861586e08f7a4db3b9a44a9ad1f0d1af => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:31:25.011345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:25.045347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:25.108714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40613e3-f96e-4b40-a925-e844fe0c8f72 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:25.141685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:25.142325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:25.142609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:25.142887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:25.207745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:25.209737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:25.241485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:25.303493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 556/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:25.309426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:25.405566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525f1fd-2764-4704-a2a2-eb33ec713039 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:25.438685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a17bc-8fe5-453c-8987-f2081622fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:25.439533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a17bc-8fe5-453c-8987-f2081622fcbf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:25.439817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a17bc-8fe5-453c-8987-f2081622fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:25.440095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a17bc-8fe5-453c-8987-f2081622fcbf None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:25.441293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a17bc-8fe5-453c-8987-f2081622fcbf None admin] RBAC: Authorizing `identity:get_project(project_id=0c50520adc4e4465b5d154ee9b201a51)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:25.442741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a17bc-8fe5-453c-8987-f2081622fcbf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:25.444491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 560/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:25 2022] GET /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:25.450486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:25.554940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0df09e-fd36-4ca4-9d1e-dc8f941f0c1e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:25.586933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb24a-2e19-437b-95e6-d8bd50025cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:25.587577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb24a-2e19-437b-95e6-d8bd50025cbc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:25.587860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb24a-2e19-437b-95e6-d8bd50025cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:25.588138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb24a-2e19-437b-95e6-d8bd50025cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:25.588989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb24a-2e19-437b-95e6-d8bd50025cbc None admin] RBAC: Adding query filter params (role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=0c50520adc4e4465b5d154ee9b201a51, user.id=861586e08f7a4db3b9a44a9ad1f0d1af) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:25.589303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb24a-2e19-437b-95e6-d8bd50025cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:25.590738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb24a-2e19-437b-95e6-d8bd50025cbc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:25.593483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:25.596253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 557/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:25 2022] GET /identity/v3/role_assignments?user.id=861586e08f7a4db3b9a44a9ad1f0d1af&scope.project.id=0c50520adc4e4465b5d154ee9b201a51&role.id=4182f79f5cda459497c049f3f07a1713 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:27.601738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc97dc-820d-473b-9de2-fc677f4ac123 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:27.602296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc97dc-820d-473b-9de2-fc677f4ac123 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:27.602296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc97dc-820d-473b-9de2-fc677f4ac123 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:27.603655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 561/1118] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:27.608829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec3f3d-7b6f-4f3d-9920-2969fe7ed494 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:27.609211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec3f3d-7b6f-4f3d-9920-2969fe7ed494 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:27.609389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec3f3d-7b6f-4f3d-9920-2969fe7ed494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:27.626406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-6eec3f3d-7b6f-4f3d-9920-2969fe7ed494 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:27.627590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6eec3f3d-7b6f-4f3d-9920-2969fe7ed494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:27.695946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 558/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:27.719197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc97dc-820d-473b-9de2-fc677f4ac123 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:27.768616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:27.818163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc97dc-820d-473b-9de2-fc677f4ac123 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:27.851450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:27.852340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:27.852605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:27.852884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] PATH_INFO: `/v3/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:27.854341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] RBAC: Authorizing `identity:get_role(role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:27.855697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:27.857103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 562/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:27 2022] GET /identity/v3/roles/4182f79f5cda459497c049f3f07a1713 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:27.862907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec3f3d-7b6f-4f3d-9920-2969fe7ed494 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:27.957862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec3f3d-7b6f-4f3d-9920-2969fe7ed494 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:27.985899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:27.992579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:27.993461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:27.997683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:27.997683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] PATH_INFO: `/v3/users/861586e08f7a4db3b9a44a9ad1f0d1af` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:27.997683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] RBAC: Authorizing `identity:get_user(user_id=861586e08f7a4db3b9a44a9ad1f0d1af)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:27.997683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:27.998250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 559/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:27 2022] GET /identity/v3/users/861586e08f7a4db3b9a44a9ad1f0d1af => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:28.004211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:28.100796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65fe50-bd36-4744-87f6-a16b910086b1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:28.116820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:28.133294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fb114-bb31-4cc0-a8e0-7214931f1dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:28.134141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fb114-bb31-4cc0-a8e0-7214931f1dce None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:28.134427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fb114-bb31-4cc0-a8e0-7214931f1dce None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:28.134702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fb114-bb31-4cc0-a8e0-7214931f1dce None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:28.135855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981fb114-bb31-4cc0-a8e0-7214931f1dce None admin] RBAC: Authorizing `identity:get_project(project_id=0c50520adc4e4465b5d154ee9b201a51)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:28.137348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981fb114-bb31-4cc0-a8e0-7214931f1dce None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:28.139022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 563/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:28 2022] GET /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:28.145377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:28.240604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bf780-0957-41d8-a06a-b25040b752a4 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:28.273610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7496803-2a0c-47f9-b6fd-8e8cfd1a3b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:28.274469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496803-2a0c-47f9-b6fd-8e8cfd1a3b16 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:28.274749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496803-2a0c-47f9-b6fd-8e8cfd1a3b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:28.275021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496803-2a0c-47f9-b6fd-8e8cfd1a3b16 None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51/users/861586e08f7a4db3b9a44a9ad1f0d1af/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:28.277272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7496803-2a0c-47f9-b6fd-8e8cfd1a3b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c50520adc4e4465b5d154ee9b201a51, user_id=861586e08f7a4db3b9a44a9ad1f0d1af, role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:28.278690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7496803-2a0c-47f9-b6fd-8e8cfd1a3b16 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:28.346362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 560/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:31:28 2022] PUT /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51/users/861586e08f7a4db3b9a44a9ad1f0d1af/roles/4182f79f5cda459497c049f3f07a1713 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:31:30.319221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8a67b-10e4-40a2-8463-731486939d98 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:30.319769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8a67b-10e4-40a2-8463-731486939d98 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:30.319769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8a67b-10e4-40a2-8463-731486939d98 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:30.321164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 564/1124] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:30.326392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40488a6-6ca0-4253-b121-feeb36edb727 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:30.326773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40488a6-6ca0-4253-b121-feeb36edb727 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:30.326949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40488a6-6ca0-4253-b121-feeb36edb727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:30.343115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a40488a6-6ca0-4253-b121-feeb36edb727 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:30.344295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a40488a6-6ca0-4253-b121-feeb36edb727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:30.395960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:30.409261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 561/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:30.432279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8a67b-10e4-40a2-8463-731486939d98 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:30.529498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8a67b-10e4-40a2-8463-731486939d98 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:30.562201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:30.563033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:30.563318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:30.563597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] PATH_INFO: `/v3/roles/4182f79f5cda459497c049f3f07a1713` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:30.565075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] RBAC: Authorizing `identity:get_role(role_id=4182f79f5cda459497c049f3f07a1713)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:30.566440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:30.567839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 565/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:30 2022] GET /identity/v3/roles/4182f79f5cda459497c049f3f07a1713 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:30.573489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40488a6-6ca0-4253-b121-feeb36edb727 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:30.668378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40488a6-6ca0-4253-b121-feeb36edb727 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:30.700812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675063be-055c-4588-8042-02aa186f787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:30.701668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675063be-055c-4588-8042-02aa186f787f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:30.701956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675063be-055c-4588-8042-02aa186f787f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:30.702233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675063be-055c-4588-8042-02aa186f787f None admin] PATH_INFO: `/v3/users/861586e08f7a4db3b9a44a9ad1f0d1af` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:30.703422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675063be-055c-4588-8042-02aa186f787f None admin] RBAC: Authorizing `identity:get_user(user_id=861586e08f7a4db3b9a44a9ad1f0d1af)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:30.704879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675063be-055c-4588-8042-02aa186f787f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:30.706475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 562/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:30 2022] GET /identity/v3/users/861586e08f7a4db3b9a44a9ad1f0d1af => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:30.712041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:30.762417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:30.807247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da48e45-a910-4825-8a4c-b3176f75b1ab None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:30.838865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:30.839491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:30.839776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:30.840055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:30.903897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:30.905355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:30.954158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:30.999737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 566/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:31.005639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675063be-055c-4588-8042-02aa186f787f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:31.060184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:31.102529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675063be-055c-4588-8042-02aa186f787f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:31.136063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3323-03a1-4219-b14b-d3e3c945bc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:31.136964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3323-03a1-4219-b14b-d3e3c945bc09 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:31.137252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3323-03a1-4219-b14b-d3e3c945bc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:31.137533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3323-03a1-4219-b14b-d3e3c945bc09 None admin] PATH_INFO: `/v3/projects/0c50520adc4e4465b5d154ee9b201a51` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:31.138710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3323-03a1-4219-b14b-d3e3c945bc09 None admin] RBAC: Authorizing `identity:get_project(project_id=0c50520adc4e4465b5d154ee9b201a51)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:31.140149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3323-03a1-4219-b14b-d3e3c945bc09 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:31.141892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 563/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:31 2022] GET /identity/v3/projects/0c50520adc4e4465b5d154ee9b201a51 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:31.148726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:31.243648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3857c2-fb50-46b0-9847-51f747622dec None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:31.276934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2051d8-32a1-49c4-9613-da612dbe4497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:31.277575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2051d8-32a1-49c4-9613-da612dbe4497 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:31.277861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2051d8-32a1-49c4-9613-da612dbe4497 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:31.278141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2051d8-32a1-49c4-9613-da612dbe4497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:31.278967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2051d8-32a1-49c4-9613-da612dbe4497 None admin] RBAC: Adding query filter params (role.id=4182f79f5cda459497c049f3f07a1713, scope.project.id=0c50520adc4e4465b5d154ee9b201a51, user.id=861586e08f7a4db3b9a44a9ad1f0d1af) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:31.279280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2051d8-32a1-49c4-9613-da612dbe4497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:31.280736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2051d8-32a1-49c4-9613-da612dbe4497 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:31.287194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 567/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:31 2022] GET /identity/v3/role_assignments?user.id=861586e08f7a4db3b9a44a9ad1f0d1af&scope.project.id=0c50520adc4e4465b5d154ee9b201a51&role.id=4182f79f5cda459497c049f3f07a1713 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:33.324737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dc6d1-9066-48f0-8658-bbd70cfc104a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:33.325271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dc6d1-9066-48f0-8658-bbd70cfc104a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:33.325271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dc6d1-9066-48f0-8658-bbd70cfc104a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:33.326664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 564/1131] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:33.331895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74f4c-cbcf-4287-9692-c1a0a6f7ffd6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:33.332310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74f4c-cbcf-4287-9692-c1a0a6f7ffd6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:33.332495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74f4c-cbcf-4287-9692-c1a0a6f7ffd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:33.348773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-29b74f4c-cbcf-4287-9692-c1a0a6f7ffd6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:33.349892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-29b74f4c-cbcf-4287-9692-c1a0a6f7ffd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:33.360158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:33.414398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 568/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:33.437624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dc6d1-9066-48f0-8658-bbd70cfc104a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:33.545115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dc6d1-9066-48f0-8658-bbd70cfc104a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:33.568325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:33.578545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec054b-8022-40f5-a79c-5c90a42690d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:33.579329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec054b-8022-40f5-a79c-5c90a42690d2 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:33.579737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec054b-8022-40f5-a79c-5c90a42690d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:33.579876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec054b-8022-40f5-a79c-5c90a42690d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:33.581064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec054b-8022-40f5-a79c-5c90a42690d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:33.582496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec054b-8022-40f5-a79c-5c90a42690d2 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:33.584170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 565/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:33.590157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b74f4c-cbcf-4287-9692-c1a0a6f7ffd6 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:33.687849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b74f4c-cbcf-4287-9692-c1a0a6f7ffd6 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:33.721611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5eaf54-587b-41f9-850e-cc075120f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:33.722258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eaf54-587b-41f9-850e-cc075120f7af None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:33.722542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eaf54-587b-41f9-850e-cc075120f7af None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:33.722815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eaf54-587b-41f9-850e-cc075120f7af None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:33.723513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5eaf54-587b-41f9-850e-cc075120f7af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:33.725845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5eaf54-587b-41f9-850e-cc075120f7af None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:33.836071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:33.842967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 569/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:31:33 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:35.931361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e45e23-fab9-47db-b8ec-87fb0318b43e None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:35.931912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e45e23-fab9-47db-b8ec-87fb0318b43e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:35.931912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e45e23-fab9-47db-b8ec-87fb0318b43e None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:35.933298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 566/1135] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:35.938408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dca506-4d7f-4f44-a47e-e18e005919f8 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:35.938771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dca506-4d7f-4f44-a47e-e18e005919f8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:35.938975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dca506-4d7f-4f44-a47e-e18e005919f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:35.955776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-55dca506-4d7f-4f44-a47e-e18e005919f8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:35.956992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-55dca506-4d7f-4f44-a47e-e18e005919f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:36.021763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 570/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:36.046391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e45e23-fab9-47db-b8ec-87fb0318b43e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:36.143227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e45e23-fab9-47db-b8ec-87fb0318b43e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:36.175691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20fccc-1d60-4f0c-b944-09b755cfabaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:36.176487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20fccc-1d60-4f0c-b944-09b755cfabaa None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:36.176775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20fccc-1d60-4f0c-b944-09b755cfabaa None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:36.177053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20fccc-1d60-4f0c-b944-09b755cfabaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:36.178213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20fccc-1d60-4f0c-b944-09b755cfabaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:36.179639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20fccc-1d60-4f0c-b944-09b755cfabaa None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:36.181253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 567/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:31:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:36.187025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dca506-4d7f-4f44-a47e-e18e005919f8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:36.281676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dca506-4d7f-4f44-a47e-e18e005919f8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:36.307342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:36.314771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b33b6-d483-4deb-ad09-2e0124698eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:36.315437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b33b6-d483-4deb-ad09-2e0124698eb5 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:36.315725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b33b6-d483-4deb-ad09-2e0124698eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:36.316007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b33b6-d483-4deb-ad09-2e0124698eb5 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:36.316745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b33b6-d483-4deb-ad09-2e0124698eb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:36.318713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b33b6-d483-4deb-ad09-2e0124698eb5 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:36.409691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 571/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:31:36 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:38.589007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c9580-ee9b-4582-b627-26cdb4b552df None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:38.589557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c9580-ee9b-4582-b627-26cdb4b552df None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:38.589557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c9580-ee9b-4582-b627-26cdb4b552df None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:38.590935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 568/1139] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:38.596175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a07a4-e5e3-421b-be19-00f3c91c8238 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:38.596560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a07a4-e5e3-421b-be19-00f3c91c8238 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:38.596783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a07a4-e5e3-421b-be19-00f3c91c8238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:38.613055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-258a07a4-e5e3-421b-be19-00f3c91c8238 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:38.614731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-258a07a4-e5e3-421b-be19-00f3c91c8238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:38.679065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 572/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:38.702998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c9580-ee9b-4582-b627-26cdb4b552df None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:38.761215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:38.800887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c9580-ee9b-4582-b627-26cdb4b552df None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:38.832656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:38.833946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:38.834230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:38.834507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:38.844239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:38.852344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:38.853545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:38.854620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 569/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:38.859438 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a07a4-e5e3-421b-be19-00f3c91c8238 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:38.954484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a07a4-e5e3-421b-be19-00f3c91c8238 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:38.968963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:38.987706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:38.988407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:38.988726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:38.989028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:38.989774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:38.990082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:38.991436 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:38.999195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 573/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:31:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:39.005405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:39.196871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361e13b-f63d-4f8b-97a3-2f0bc19a6dab None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:39.215934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:39.229296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:39.230150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:39.230430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:39.230701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] PATH_INFO: `/v3/users/94d685a333a24981b3c12489427dd835` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:39.243194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] RBAC: Authorizing `identity:get_user(user_id=94d685a333a24981b3c12489427dd835)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:39.244639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:39.246500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 570/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:39 2022] GET /identity/v3/users/94d685a333a24981b3c12489427dd835 => generated 286 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:39.252044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:39.347391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b305-f34e-4a59-af5f-9fdc13099a70 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:39.379149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:39.380273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:39.380591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:39.380875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:39.422838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:39.444886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:39.446343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:39.539904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 574/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:39.545784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:39.641872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b5a0-9ea4-4ab9-954a-59a43cbbc27e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:39.674582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48786c79-4d02-498a-836d-66f97d72abe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:39.675418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48786c79-4d02-498a-836d-66f97d72abe5 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:39.675704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48786c79-4d02-498a-836d-66f97d72abe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:39.675980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48786c79-4d02-498a-836d-66f97d72abe5 None admin] PATH_INFO: `/v3/projects/9569098b0b5a49c08352d082952182b9` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:39.677710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48786c79-4d02-498a-836d-66f97d72abe5 None admin] RBAC: Authorizing `identity:get_project(project_id=9569098b0b5a49c08352d082952182b9)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:39.679144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48786c79-4d02-498a-836d-66f97d72abe5 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:39.680847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 571/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:39 2022] GET /identity/v3/projects/9569098b0b5a49c08352d082952182b9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:39.686717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:39.761729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:39.783871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6be267-eb2a-47b5-b512-b4b0c6a99ec2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:39.817476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00221a11-ac72-458c-953b-b9f7068446a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:39.818118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221a11-ac72-458c-953b-b9f7068446a7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:39.818400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221a11-ac72-458c-953b-b9f7068446a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:39.818674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221a11-ac72-458c-953b-b9f7068446a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:39.819493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00221a11-ac72-458c-953b-b9f7068446a7 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=9569098b0b5a49c08352d082952182b9, user.id=94d685a333a24981b3c12489427dd835) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:39.819802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00221a11-ac72-458c-953b-b9f7068446a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:39.821252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00221a11-ac72-458c-953b-b9f7068446a7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:39.826458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 575/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:39 2022] GET /identity/v3/role_assignments?user.id=94d685a333a24981b3c12489427dd835&scope.project.id=9569098b0b5a49c08352d082952182b9&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:41.818562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d65ec-86da-499e-a709-e13adc900266 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:41.819201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d65ec-86da-499e-a709-e13adc900266 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:41.819201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d65ec-86da-499e-a709-e13adc900266 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:41.820504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 572/1147] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:41.825675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2b226-6294-497c-91d1-1bb5e8eb6dac None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:41.826053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2b226-6294-497c-91d1-1bb5e8eb6dac None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:41.826236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2b226-6294-497c-91d1-1bb5e8eb6dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:41.842358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-aad2b226-6294-497c-91d1-1bb5e8eb6dac None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:41.843703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-aad2b226-6294-497c-91d1-1bb5e8eb6dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:41.906250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 576/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:41.929403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d65ec-86da-499e-a709-e13adc900266 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:41.975236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:42.026335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d65ec-86da-499e-a709-e13adc900266 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:42.059882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:42.060684 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:42.060964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:42.061232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:42.070951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:42.078397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:42.079485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:42.080389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 573/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:31:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:42.085681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2b226-6294-497c-91d1-1bb5e8eb6dac None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:42.151043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:42.180593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2b226-6294-497c-91d1-1bb5e8eb6dac None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:42.212297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:42.212971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:42.213255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:42.213527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:42.214264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:42.214567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:42.215912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:42.224129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 577/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:42.230041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:42.325408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeec44d-15be-422e-ad40-1ce5cd3f4b8a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:42.331895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:42.358258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:42.359103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:42.359386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:42.359664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] PATH_INFO: `/v3/users/94d685a333a24981b3c12489427dd835` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:42.360858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] RBAC: Authorizing `identity:get_user(user_id=94d685a333a24981b3c12489427dd835)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:42.362292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:42.364244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 574/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:42 2022] GET /identity/v3/users/94d685a333a24981b3c12489427dd835 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:42.370646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:42.466497 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68105323-1047-4519-9bcd-1c858488ea4f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:42.494377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:42.499091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f577f-ff6d-4752-a3d0-d91f38effba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:42.499944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f577f-ff6d-4752-a3d0-d91f38effba9 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:42.500227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f577f-ff6d-4752-a3d0-d91f38effba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:42.500535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f577f-ff6d-4752-a3d0-d91f38effba9 None admin] PATH_INFO: `/v3/projects/9569098b0b5a49c08352d082952182b9` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:42.501719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f577f-ff6d-4752-a3d0-d91f38effba9 None admin] RBAC: Authorizing `identity:get_project(project_id=9569098b0b5a49c08352d082952182b9)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:42.507549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f577f-ff6d-4752-a3d0-d91f38effba9 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:42.507549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 578/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:42 2022] GET /identity/v3/projects/9569098b0b5a49c08352d082952182b9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:42.510565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:42.606877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a5660-3b5f-4932-8786-8a7ebafeb0cb None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:42.639750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b561b-099b-4655-acd0-77097e2d862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:42.640659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b561b-099b-4655-acd0-77097e2d862f None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:42.640946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b561b-099b-4655-acd0-77097e2d862f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:42.641222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b561b-099b-4655-acd0-77097e2d862f None admin] PATH_INFO: `/v3/projects/9569098b0b5a49c08352d082952182b9/users/94d685a333a24981b3c12489427dd835/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:42.643152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b561b-099b-4655-acd0-77097e2d862f None admin] RBAC: Authorizing `identity:create_grant(project_id=9569098b0b5a49c08352d082952182b9, user_id=94d685a333a24981b3c12489427dd835, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:42.644628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b561b-099b-4655-acd0-77097e2d862f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:42.740195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 575/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:31:42 2022] PUT /identity/v3/projects/9569098b0b5a49c08352d082952182b9/users/94d685a333a24981b3c12489427dd835/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:31:44.732411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b458c5-6555-48ac-a3d0-1c9489675e60 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:44.732974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b458c5-6555-48ac-a3d0-1c9489675e60 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:44.732974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b458c5-6555-48ac-a3d0-1c9489675e60 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:44.734373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 579/1154] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:44.739605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161835f-ded4-4f25-890d-eef46f3c059f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:44.739998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161835f-ded4-4f25-890d-eef46f3c059f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:44.740161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161835f-ded4-4f25-890d-eef46f3c059f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:44.756372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7161835f-ded4-4f25-890d-eef46f3c059f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:44.757584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7161835f-ded4-4f25-890d-eef46f3c059f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:44.783190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:44.820832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 576/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:44.843910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b458c5-6555-48ac-a3d0-1c9489675e60 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:44.942483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b458c5-6555-48ac-a3d0-1c9489675e60 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:44.973992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:44.974761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:44.975043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:44.975313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:44.985653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:44.993663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:44.994769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:44.995827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 580/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:31:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:45.000952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161835f-ded4-4f25-890d-eef46f3c059f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:45.096120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161835f-ded4-4f25-890d-eef46f3c059f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:45.131034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:45.131695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:45.131979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:45.132255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:45.133028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:45.133338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:45.134713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:45.142529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 577/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:31:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:45.148349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:45.208573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:45.248237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aea68-4966-4476-a316-7fc883651612 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:45.280697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:45.281888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:45.281888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:45.282104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] PATH_INFO: `/v3/users/94d685a333a24981b3c12489427dd835` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:45.283537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] RBAC: Authorizing `identity:get_user(user_id=94d685a333a24981b3c12489427dd835)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:45.285002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:45.287186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 581/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:45 2022] GET /identity/v3/users/94d685a333a24981b3c12489427dd835 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:45.292949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:45.319715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:45.389105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0462-d0a0-4cdd-90bf-254a32dd4e66 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:45.423279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:45.423920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:45.424205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:45.424510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:45.488397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:45.489862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:45.515407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:45.584769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 578/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:45.590635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:45.685928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de5000-b90b-4a48-b79e-ae98988fa57e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:45.711627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:45.719024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a28f9-13c7-4d10-9ec4-a252cce4a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:45.719866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a28f9-13c7-4d10-9ec4-a252cce4a678 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:45.720976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a28f9-13c7-4d10-9ec4-a252cce4a678 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:45.720976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a28f9-13c7-4d10-9ec4-a252cce4a678 None admin] PATH_INFO: `/v3/projects/9569098b0b5a49c08352d082952182b9` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:45.721631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a28f9-13c7-4d10-9ec4-a252cce4a678 None admin] RBAC: Authorizing `identity:get_project(project_id=9569098b0b5a49c08352d082952182b9)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:45.723058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a28f9-13c7-4d10-9ec4-a252cce4a678 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:45.724799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 582/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:45 2022] GET /identity/v3/projects/9569098b0b5a49c08352d082952182b9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:45.730806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:45.839812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfd1fa-520e-454f-b172-93b82d82efda None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:45.863262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:45.871240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e80b8-3d10-4b34-8d10-e994e7cf9afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:45.892843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e80b8-3d10-4b34-8d10-e994e7cf9afb None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:45.892843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e80b8-3d10-4b34-8d10-e994e7cf9afb None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:45.892843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e80b8-3d10-4b34-8d10-e994e7cf9afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:45.892843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e80b8-3d10-4b34-8d10-e994e7cf9afb None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=9569098b0b5a49c08352d082952182b9, user.id=94d685a333a24981b3c12489427dd835) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:45.892843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e80b8-3d10-4b34-8d10-e994e7cf9afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:45.892843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e80b8-3d10-4b34-8d10-e994e7cf9afb None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:45.893369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 579/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:45 2022] GET /identity/v3/role_assignments?user.id=94d685a333a24981b3c12489427dd835&scope.project.id=9569098b0b5a49c08352d082952182b9&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:47.915648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790ce06-86c2-459d-b6c1-b41753f3277b None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:47.916203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790ce06-86c2-459d-b6c1-b41753f3277b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:47.916203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790ce06-86c2-459d-b6c1-b41753f3277b None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:47.917607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 583/1162] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:47.922898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02380148-46be-4eed-9115-7413a2b8361b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:47.923296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02380148-46be-4eed-9115-7413a2b8361b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:47.923444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02380148-46be-4eed-9115-7413a2b8361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:47.940250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-02380148-46be-4eed-9115-7413a2b8361b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:47.941453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-02380148-46be-4eed-9115-7413a2b8361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:48.004179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 580/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:48.026456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790ce06-86c2-459d-b6c1-b41753f3277b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:48.126370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790ce06-86c2-459d-b6c1-b41753f3277b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:48.158693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:48.159332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:48.159619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:48.159898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:48.217849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:48.224566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:48.226006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:48.323311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 584/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:48.329318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02380148-46be-4eed-9115-7413a2b8361b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:48.425040 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02380148-46be-4eed-9115-7413a2b8361b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:48.443943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:48.457141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f226d9-c7a1-425e-a0ad-943257b45141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:48.458459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f226d9-c7a1-425e-a0ad-943257b45141 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:48.458743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f226d9-c7a1-425e-a0ad-943257b45141 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:48.459020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f226d9-c7a1-425e-a0ad-943257b45141 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:48.465447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f226d9-c7a1-425e-a0ad-943257b45141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:48.466890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f226d9-c7a1-425e-a0ad-943257b45141 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-45f226d9-c7a1-425e-a0ad-943257b45141 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:48.481988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:48.483174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:31:48.484378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 581/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:31:48 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:48.488989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:48.580761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:48.586916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fcea1-e716-4f56-9b8d-08ed994929f7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:48.623525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb98b-c641-4e9b-a1d0-acd82fcd859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:48.624181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb98b-c641-4e9b-a1d0-acd82fcd859c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:48.624489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb98b-c641-4e9b-a1d0-acd82fcd859c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:48.624771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb98b-c641-4e9b-a1d0-acd82fcd859c None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:48.625489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cb98b-c641-4e9b-a1d0-acd82fcd859c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:48.625799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cb98b-c641-4e9b-a1d0-acd82fcd859c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:48.627158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cb98b-c641-4e9b-a1d0-acd82fcd859c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:48.634572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 585/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Jul 20 17:31:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:50.647782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd2cd-0068-4d0d-b19a-f90368c56598 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:50.648302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd2cd-0068-4d0d-b19a-f90368c56598 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:50.648576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd2cd-0068-4d0d-b19a-f90368c56598 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:50.649743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 582/1167] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:50.654819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a2451-3a70-4230-be88-3fcfe2b0c034 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:50.655247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a2451-3a70-4230-be88-3fcfe2b0c034 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:50.655377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a2451-3a70-4230-be88-3fcfe2b0c034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:50.671712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b30a2451-3a70-4230-be88-3fcfe2b0c034 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:50.672913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b30a2451-3a70-4230-be88-3fcfe2b0c034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:50.735736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 586/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 17:31:50.759220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbd2cd-0068-4d0d-b19a-f90368c56598 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:50.857035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbd2cd-0068-4d0d-b19a-f90368c56598 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:50.893508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97930216-674d-4c1f-9b7d-d27181924870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:50.894160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97930216-674d-4c1f-9b7d-d27181924870 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:50.894442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97930216-674d-4c1f-9b7d-d27181924870 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:50.894718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97930216-674d-4c1f-9b7d-d27181924870 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:50.896086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97930216-674d-4c1f-9b7d-d27181924870 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:50.897478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97930216-674d-4c1f-9b7d-d27181924870 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:50.981664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 583/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Wed Jul 20 17:31:50 2022] POST /identity/v3/domains => generated 267 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:53.016128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9795638-df5a-412f-bfda-53f53a825f98 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:53.016705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9795638-df5a-412f-bfda-53f53a825f98 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:53.016705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9795638-df5a-412f-bfda-53f53a825f98 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:53.018103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 587/1170] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:53.023618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894f70-674e-43c3-a5ff-3c4295fe5bd7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:53.023992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894f70-674e-43c3-a5ff-3c4295fe5bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:53.024168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894f70-674e-43c3-a5ff-3c4295fe5bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:53.041136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-27894f70-674e-43c3-a5ff-3c4295fe5bd7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:53.042303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-27894f70-674e-43c3-a5ff-3c4295fe5bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:53.075186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:53.110850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 584/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:53.133783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9795638-df5a-412f-bfda-53f53a825f98 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:53.221048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:53.232065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9795638-df5a-412f-bfda-53f53a825f98 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:53.264776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766f843-c43b-4c5e-912c-597a2adf7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:53.265627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766f843-c43b-4c5e-912c-597a2adf7840 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:53.265911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766f843-c43b-4c5e-912c-597a2adf7840 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:53.266233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766f843-c43b-4c5e-912c-597a2adf7840 None admin] PATH_INFO: `/v3/domains/1c5819223e4347979712d14b8ca759e5` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:53.276664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766f843-c43b-4c5e-912c-597a2adf7840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1c5819223e4347979712d14b8ca759e5)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:53.278093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766f843-c43b-4c5e-912c-597a2adf7840 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:53.279710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 588/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Jul 20 17:31:53 2022] GET /identity/v3/domains/1c5819223e4347979712d14b8ca759e5 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:53.285898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894f70-674e-43c3-a5ff-3c4295fe5bd7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:53.378838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894f70-674e-43c3-a5ff-3c4295fe5bd7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:53.412020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea22f3-0bf4-4b71-ae4b-f366f867aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:53.412678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea22f3-0bf4-4b71-ae4b-f366f867aa29 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:53.412963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea22f3-0bf4-4b71-ae4b-f366f867aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:53.413235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea22f3-0bf4-4b71-ae4b-f366f867aa29 None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:53.413940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea22f3-0bf4-4b71-ae4b-f366f867aa29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:53.415359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea22f3-0bf4-4b71-ae4b-f366f867aa29 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:53.532404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 585/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:31:53 2022] POST /identity/v3/projects => generated 361 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:55.579612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28855dab-b135-4576-b653-84448b0298ac None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:55.580164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28855dab-b135-4576-b653-84448b0298ac None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:55.580164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28855dab-b135-4576-b653-84448b0298ac None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:55.581580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 589/1174] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:55.587322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25726-8a92-44f0-adb2-23959d16b260 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:55.587699 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25726-8a92-44f0-adb2-23959d16b260 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:55.587879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25726-8a92-44f0-adb2-23959d16b260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:55.604093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cda25726-8a92-44f0-adb2-23959d16b260 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:55.605296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cda25726-8a92-44f0-adb2-23959d16b260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:55.621127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:55.669566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 586/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:55.694330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28855dab-b135-4576-b653-84448b0298ac None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:55.791597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28855dab-b135-4576-b653-84448b0298ac None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:55.823785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1438f7d-2d3c-4092-9ff0-bcb88c5145ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:55.824653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1438f7d-2d3c-4092-9ff0-bcb88c5145ba None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:55.824940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1438f7d-2d3c-4092-9ff0-bcb88c5145ba None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:55.825216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1438f7d-2d3c-4092-9ff0-bcb88c5145ba None admin] PATH_INFO: `/v3/domains/1c5819223e4347979712d14b8ca759e5` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:55.826383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1438f7d-2d3c-4092-9ff0-bcb88c5145ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=1c5819223e4347979712d14b8ca759e5)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:55.827805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1438f7d-2d3c-4092-9ff0-bcb88c5145ba None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:55.829423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 590/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Jul 20 17:31:55 2022] GET /identity/v3/domains/1c5819223e4347979712d14b8ca759e5 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:55.835230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25726-8a92-44f0-adb2-23959d16b260 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:55.941276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25726-8a92-44f0-adb2-23959d16b260 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:55.973206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05dd69-7f5f-42d8-8454-d09d7a89d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:55.973858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05dd69-7f5f-42d8-8454-d09d7a89d33d None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:55.974139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05dd69-7f5f-42d8-8454-d09d7a89d33d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:55.974414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05dd69-7f5f-42d8-8454-d09d7a89d33d None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:55.975111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05dd69-7f5f-42d8-8454-d09d7a89d33d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:55.976563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05dd69-7f5f-42d8-8454-d09d7a89d33d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:56.066138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 587/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:31:55 2022] POST /identity/v3/users => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:31:58.147363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc55c3-1dd9-4547-a14b-250a6863d6e0 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:58.147855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc55c3-1dd9-4547-a14b-250a6863d6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:58.147966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc55c3-1dd9-4547-a14b-250a6863d6e0 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:58.149335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 591/1178] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:31:58.154468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ffae3-ebe3-4d37-9ac5-5467a88be9d4 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:58.154880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ffae3-ebe3-4d37-9ac5-5467a88be9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:58.155026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ffae3-ebe3-4d37-9ac5-5467a88be9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:58.172105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e61ffae3-ebe3-4d37-9ac5-5467a88be9d4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:31:58.173327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e61ffae3-ebe3-4d37-9ac5-5467a88be9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:31:58.175235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:58.244019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 588/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:31:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:31:58.267782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc55c3-1dd9-4547-a14b-250a6863d6e0 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:58.286831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:58.366669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc55c3-1dd9-4547-a14b-250a6863d6e0 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:58.398626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:58.399410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:58.399697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:58.399978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:58.411607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:31:58.419199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:31:58.420367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:31:58.421491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 592/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:31:58.427028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ffae3-ebe3-4d37-9ac5-5467a88be9d4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:58.523462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ffae3-ebe3-4d37-9ac5-5467a88be9d4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:58.556452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:58.557129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:58.557416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:58.557702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:58.558443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:58.558753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:58.560123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:58.564945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:58.568710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 589/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:58.574448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:58.649476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:58.669733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6007c5-fbe4-4f56-bd5d-5da542cc256a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:58.702091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:58.702936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:58.703217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:58.703492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] PATH_INFO: `/v3/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:58.716048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] RBAC: Authorizing `identity:get_user(user_id=ae33ad0da3eb4eb6b1931d9cc96ea0ed)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:58.717503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:58.719113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 593/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:31:58 2022] GET /identity/v3/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:58.725254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:58.821768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc25e9-4d2f-4216-8f5f-732b4e86aa7c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:58.853188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:58.853825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:58.854113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:58.854392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:58.916678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:58.918901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:58.920341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:59.014483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 590/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:31:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:31:59.020319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:59.117968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd358-0a36-4619-9c94-ecef79c298b8 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:59.149775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4648f6-57f7-4abb-920a-55e17df10830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:59.150619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4648f6-57f7-4abb-920a-55e17df10830 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:59.150899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4648f6-57f7-4abb-920a-55e17df10830 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:59.151174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4648f6-57f7-4abb-920a-55e17df10830 None admin] PATH_INFO: `/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:59.152326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4648f6-57f7-4abb-920a-55e17df10830 None admin] RBAC: Authorizing `identity:get_project(project_id=24c92bfc0f9a4553a1ae75ebe893636a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:59.153765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4648f6-57f7-4abb-920a-55e17df10830 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:59.155424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 594/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:31:59 2022] GET /identity/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:31:59.162284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:31:59.256345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:31:59.259659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee74bf-c5f7-483a-b009-bd8e77cef8e8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:31:59.292788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc03bc-5037-4938-9812-01f836a1134d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:31:59.293432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc03bc-5037-4938-9812-01f836a1134d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:31:59.293718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc03bc-5037-4938-9812-01f836a1134d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:31:59.293994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc03bc-5037-4938-9812-01f836a1134d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:31:59.294818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc03bc-5037-4938-9812-01f836a1134d None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=24c92bfc0f9a4553a1ae75ebe893636a, user.id=ae33ad0da3eb4eb6b1931d9cc96ea0ed) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:31:59.295133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc03bc-5037-4938-9812-01f836a1134d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:31:59.296598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc03bc-5037-4938-9812-01f836a1134d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:31:59.301955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 591/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:31:59 2022] GET /identity/v3/role_assignments?user.id=ae33ad0da3eb4eb6b1931d9cc96ea0ed&scope.project.id=24c92bfc0f9a4553a1ae75ebe893636a&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:01.270898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8efbef-7159-4b4e-b579-d2df03f34139 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:01.271444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8efbef-7159-4b4e-b579-d2df03f34139 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:01.271444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8efbef-7159-4b4e-b579-d2df03f34139 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:01.272889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 595/1186] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:32:01.278144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e152-eeb3-487e-9059-ce5c5c5cb16e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:01.278552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e152-eeb3-487e-9059-ce5c5c5cb16e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:01.278700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e152-eeb3-487e-9059-ce5c5c5cb16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:01.295876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4f03e152-eeb3-487e-9059-ce5c5c5cb16e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:32:01.297099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4f03e152-eeb3-487e-9059-ce5c5c5cb16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:32:01.363467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 592/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:32:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:32:01.386043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8efbef-7159-4b4e-b579-d2df03f34139 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:01.389783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:01.484476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8efbef-7159-4b4e-b579-d2df03f34139 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:01.516206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:01.517029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:01.517311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:01.517582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:01.528171 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:32:01.535930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:32:01.537111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:32:01.537980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 596/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:32:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:32:01.542951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e152-eeb3-487e-9059-ce5c5c5cb16e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:01.631380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:01.637747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e152-eeb3-487e-9059-ce5c5c5cb16e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:01.671110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:01.671768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:01.672057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:01.672337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:01.673111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:32:01.673422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:01.674795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:01.682715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 593/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:01.688458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:01.747776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:01.784928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d47e-245a-41ff-af1a-3abf0eedac4a None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:01.817697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:01.818545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:01.818825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:01.819096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] PATH_INFO: `/v3/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:01.820252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] RBAC: Authorizing `identity:get_user(user_id=ae33ad0da3eb4eb6b1931d9cc96ea0ed)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:01.821690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:01.823276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 597/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:32:01 2022] GET /identity/v3/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:01.828853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:01.928119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8c6d1-1090-4674-95ea-1812f8df24bb None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:01.960230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b788f-a7ca-4d87-8860-b3fa2f7d0c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:01.961111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b788f-a7ca-4d87-8860-b3fa2f7d0c92 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:01.961396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b788f-a7ca-4d87-8860-b3fa2f7d0c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:01.961673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b788f-a7ca-4d87-8860-b3fa2f7d0c92 None admin] PATH_INFO: `/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:01.962843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b788f-a7ca-4d87-8860-b3fa2f7d0c92 None admin] RBAC: Authorizing `identity:get_project(project_id=24c92bfc0f9a4553a1ae75ebe893636a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:01.964280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b788f-a7ca-4d87-8860-b3fa2f7d0c92 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:01.965988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 594/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:32:01 2022] GET /identity/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:01.972088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:02.073056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3491b-fd69-4a97-801b-fd8bbb40311f None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:02.100812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:02.105763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed27a1-3b2d-48cb-9c8d-9ab17fbedebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:02.106652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed27a1-3b2d-48cb-9c8d-9ab17fbedebd None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:02.106939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed27a1-3b2d-48cb-9c8d-9ab17fbedebd None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:02.107215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed27a1-3b2d-48cb-9c8d-9ab17fbedebd None admin] PATH_INFO: `/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:02.109159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed27a1-3b2d-48cb-9c8d-9ab17fbedebd None admin] RBAC: Authorizing `identity:create_grant(project_id=24c92bfc0f9a4553a1ae75ebe893636a, user_id=ae33ad0da3eb4eb6b1931d9cc96ea0ed, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:02.110588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed27a1-3b2d-48cb-9c8d-9ab17fbedebd None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:02.211682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 598/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:32:01 2022] PUT /identity/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:32:04.203210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1aac2-5734-477e-a60a-9c3097a146b9 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:04.203721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1aac2-5734-477e-a60a-9c3097a146b9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:04.203832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1aac2-5734-477e-a60a-9c3097a146b9 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:04.205174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 595/1193] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:32:04.210947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dcff7-d598-4fd6-b99a-3f252197f665 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:04.211360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dcff7-d598-4fd6-b99a-3f252197f665 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:04.211516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dcff7-d598-4fd6-b99a-3f252197f665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:04.228701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-120dcff7-d598-4fd6-b99a-3f252197f665 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:32:04.229885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-120dcff7-d598-4fd6-b99a-3f252197f665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:32:04.294392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 599/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:32:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:32:04.318053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1aac2-5734-477e-a60a-9c3097a146b9 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:04.342396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:04.423803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1aac2-5734-477e-a60a-9c3097a146b9 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:04.456927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:04.457708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:04.457996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:04.458272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:04.467981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:32:04.476050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:32:04.477157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:32:04.478233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 596/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:32:04.483381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120dcff7-d598-4fd6-b99a-3f252197f665 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:04.579366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120dcff7-d598-4fd6-b99a-3f252197f665 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:04.601183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:04.612088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:04.618866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:04.618866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:04.618866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:04.618866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:32:04.618866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:04.618866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:04.624001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 600/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:04.630103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:04.714736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:04.729991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852f8c5-f073-4329-bf90-50a51ddabfd2 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:04.763325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:04.764173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:04.764474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:04.764756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] PATH_INFO: `/v3/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:04.765915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] RBAC: Authorizing `identity:get_user(user_id=ae33ad0da3eb4eb6b1931d9cc96ea0ed)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:04.767339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:04.769015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 597/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Jul 20 17:32:04 2022] GET /identity/v3/users/ae33ad0da3eb4eb6b1931d9cc96ea0ed => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:04.775378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:04.870743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b32b0-9159-4b9b-9880-e4b083f55f6d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:04.902846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:04.903473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:04.903757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:04.904034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:04.953871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:04.968890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:04.970134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:05.063849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 601/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:32:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:32:05.070264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:05.166240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73f3ac-aef2-4ff1-933c-54c0b3f0fa73 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:05.198676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db111ed-26d3-44b8-ac40-fa7b228f4bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:05.199553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db111ed-26d3-44b8-ac40-fa7b228f4bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:05.199838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db111ed-26d3-44b8-ac40-fa7b228f4bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:05.200113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db111ed-26d3-44b8-ac40-fa7b228f4bd3 None admin] PATH_INFO: `/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:05.201319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db111ed-26d3-44b8-ac40-fa7b228f4bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=24c92bfc0f9a4553a1ae75ebe893636a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:05.202764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db111ed-26d3-44b8-ac40-fa7b228f4bd3 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:05.204450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 598/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Jul 20 17:32:05 2022] GET /identity/v3/projects/24c92bfc0f9a4553a1ae75ebe893636a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:05.210508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:32:05.289894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:05.305006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd256e-edf9-4da0-9d1e-536f33bdb2cf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:32:05.337399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6676bb-b716-4643-af92-3022d0b6ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:32:05.338040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6676bb-b716-4643-af92-3022d0b6ea57 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:32:05.338324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6676bb-b716-4643-af92-3022d0b6ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:32:05.338599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6676bb-b716-4643-af92-3022d0b6ea57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:32:05.339421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6676bb-b716-4643-af92-3022d0b6ea57 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=24c92bfc0f9a4553a1ae75ebe893636a, user.id=ae33ad0da3eb4eb6b1931d9cc96ea0ed) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:32:05.339735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6676bb-b716-4643-af92-3022d0b6ea57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:32:05.341209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6676bb-b716-4643-af92-3022d0b6ea57 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:32:05.346836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 602/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:32:05 2022] GET /identity/v3/role_assignments?user.id=ae33ad0da3eb4eb6b1931d9cc96ea0ed&scope.project.id=24c92bfc0f9a4553a1ae75ebe893636a&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:32:15.194650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:32:15.344230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=26 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:20.007303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f95ea4-929b-488b-9149-feed2e3955ed None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:20.007888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f95ea4-929b-488b-9149-feed2e3955ed None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:20.007888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f95ea4-929b-488b-9149-feed2e3955ed None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:20.009275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 599/1201] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:20.014624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99915d5-40ad-4ec6-8e5b-d635d8eca1a3 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:20.015989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99915d5-40ad-4ec6-8e5b-d635d8eca1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:20.015989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99915d5-40ad-4ec6-8e5b-d635d8eca1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:20.032251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-a99915d5-40ad-4ec6-8e5b-d635d8eca1a3 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:20.033742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a99915d5-40ad-4ec6-8e5b-d635d8eca1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:20.099555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 603/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:20.124879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f95ea4-929b-488b-9149-feed2e3955ed None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:20.223965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f95ea4-929b-488b-9149-feed2e3955ed None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:20.256512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:20.257298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:20.257630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:20.257859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:20.267477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:20.269483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:20.287198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:20.288288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:20.289492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:20.289492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:20.289492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:20.289492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:20.289492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 600/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:20.294583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99915d5-40ad-4ec6-8e5b-d635d8eca1a3 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:20.389006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99915d5-40ad-4ec6-8e5b-d635d8eca1a3 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:20.416801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:20.422362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e0e9e-7a95-45de-a342-faa4903d1de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:20.424830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e0e9e-7a95-45de-a342-faa4903d1de8 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:20.424830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e0e9e-7a95-45de-a342-faa4903d1de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:20.424830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e0e9e-7a95-45de-a342-faa4903d1de8 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:20.424830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e0e9e-7a95-45de-a342-faa4903d1de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:20.424830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e0e9e-7a95-45de-a342-faa4903d1de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:20.426048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e0e9e-7a95-45de-a342-faa4903d1de8 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:20.436858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 604/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:20.443198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:20.497117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:20.542796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939b404-4872-4e4e-8803-07d91ae3275f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:20.575898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f5cde-d42c-489c-b70a-87e13fc268cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:20.576603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f5cde-d42c-489c-b70a-87e13fc268cc None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:20.576892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f5cde-d42c-489c-b70a-87e13fc268cc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:20.577168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f5cde-d42c-489c-b70a-87e13fc268cc None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:20.577881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f5cde-d42c-489c-b70a-87e13fc268cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:20.579313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f5cde-d42c-489c-b70a-87e13fc268cc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:20.695941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 601/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Jul 20 17:33:20 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:33:22.755476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7cf5f-07ab-4737-b2c3-b539ce3cb557 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:22.756079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7cf5f-07ab-4737-b2c3-b539ce3cb557 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:22.756079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7cf5f-07ab-4737-b2c3-b539ce3cb557 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:22.757450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 605/1206] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:22.762857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89a0f-8283-4ece-a8cd-a49d222d56e9 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:22.763275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89a0f-8283-4ece-a8cd-a49d222d56e9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:22.763407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89a0f-8283-4ece-a8cd-a49d222d56e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:22.780947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e7c89a0f-8283-4ece-a8cd-a49d222d56e9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:22.782128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e7c89a0f-8283-4ece-a8cd-a49d222d56e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:22.847655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 602/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:22.870978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7cf5f-07ab-4737-b2c3-b539ce3cb557 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:22.970967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7cf5f-07ab-4737-b2c3-b539ce3cb557 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:23.003339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:23.004123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:23.004412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:23.004709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:23.017982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:33:23.024351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:23.025550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:23.026677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 606/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:33:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:23.031545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89a0f-8283-4ece-a8cd-a49d222d56e9 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:23.069981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:23.129708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89a0f-8283-4ece-a8cd-a49d222d56e9 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:23.163272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:23.163933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:23.164217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:23.164521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:23.165265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:23.165574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:23.166945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:23.175634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 603/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:33:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:23.181635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:23.230495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:23.278343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339aa5f-d3f2-49f4-af4d-9d4feb2b581e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:23.311073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:23.311862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:23.312148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:23.312426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:23.322659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:23.324517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:23.341903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:23.347552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:23.348487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:23.348487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:23.348487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:23.348487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:23.348487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 607/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:23.348908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:23.445173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:23.447979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7cdb-92f0-43b2-bcaa-336c2c4b215f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:23.480334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:23.481035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:23.481318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:23.481593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:23.482303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:23.482609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:23.483962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:23.495073 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 604/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:23.500650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:23.598372 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f065de8-d190-423f-8f6a-3363f1c61df7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:23.599786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:23.631241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:23.632041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:23.632405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:23.635482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:23.640087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:23.641569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:33:23.655689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:23.660085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:23.661408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 608/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:33:23.663126 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:23.770897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cc9e5-c1f1-4f15-a84c-d753c8ee2321 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:23.804022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:23.804710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:23.804994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:23.805269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:23.806104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:23.806416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:23.807931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:23.825813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 605/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jul 20 17:33:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:23.832056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:23.929311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2c98e-43df-4878-aa8f-7cade4b95728 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:23.962543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:23.963186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:23.963471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:23.963748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:23.991980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:24.029979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:24.031426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:24.134106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 609/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:33:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:33:24.139921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:24.158456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:24.237679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054017a-412d-40b5-8c13-bf5968ff7fbf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:24.270732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:24.271516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:24.271823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:24.272093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:24.282162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:24.283616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:24.301314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:24.302386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:24.303519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:24.303519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:24.303519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:24.303519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:24.303519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 606/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:24.308382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:24.368295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:24.405017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ed69a-f2be-4b85-88ba-9b2e4fb31894 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:24.437227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:24.437877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:24.438159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:24.438435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:24.439145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:24.439451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:24.441378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:24.452449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 610/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:24.458333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:24.526604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:24.556246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379372d3-b3f8-45aa-bf42-436971c9abec None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:24.588499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:24.589289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:24.589573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:24.589847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:24.596251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:24.598244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:33:24.611774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:24.612912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:24.617276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 607/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:33:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:24.621825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:24.724012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020917d4-1f58-4796-beb7-c3fc89eec300 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:24.745069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:24.756544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eb5ca-36f9-4cf3-b105-dc55371ab98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:24.757241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eb5ca-36f9-4cf3-b105-dc55371ab98c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:24.757530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eb5ca-36f9-4cf3-b105-dc55371ab98c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:24.757809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eb5ca-36f9-4cf3-b105-dc55371ab98c None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:24.758592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eb5ca-36f9-4cf3-b105-dc55371ab98c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:24.758902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eb5ca-36f9-4cf3-b105-dc55371ab98c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:24.760323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eb5ca-36f9-4cf3-b105-dc55371ab98c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:24.772479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 611/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:33:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:24.778824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:24.879141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef2664-8d23-4a14-bd49-18d7d0d60798 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:24.906849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:24.913330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab524e2-bd0d-457c-84a6-412783a03ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:24.913981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab524e2-bd0d-457c-84a6-412783a03ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:24.914268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab524e2-bd0d-457c-84a6-412783a03ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:24.914550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab524e2-bd0d-457c-84a6-412783a03ba4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:24.915379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab524e2-bd0d-457c-84a6-412783a03ba4 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=4871b13f87f1498ba33f8ecd2a6c0e39) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:24.915696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab524e2-bd0d-457c-84a6-412783a03ba4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:24.917436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab524e2-bd0d-457c-84a6-412783a03ba4 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:24.922770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 608/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:33:24 2022] GET /identity/v3/role_assignments?user.id=4871b13f87f1498ba33f8ecd2a6c0e39&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:26.928987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba6310-4224-488e-823a-eebc94123b3c None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:26.929569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba6310-4224-488e-823a-eebc94123b3c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:26.929569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba6310-4224-488e-823a-eebc94123b3c None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:26.930896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 612/1220] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:26.936001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261bd23-a860-40f3-90da-e740101aee7c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:26.936511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261bd23-a860-40f3-90da-e740101aee7c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:26.936610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261bd23-a860-40f3-90da-e740101aee7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:26.953685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d261bd23-a860-40f3-90da-e740101aee7c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:26.954883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d261bd23-a860-40f3-90da-e740101aee7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:27.019618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 609/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:27.042911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba6310-4224-488e-823a-eebc94123b3c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:27.140827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba6310-4224-488e-823a-eebc94123b3c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:27.173718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:27.175105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:27.175397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:27.175669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:27.185589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:33:27.193249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:27.194445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:27.195515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 613/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:33:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:27.200843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261bd23-a860-40f3-90da-e740101aee7c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:27.297195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261bd23-a860-40f3-90da-e740101aee7c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:27.320378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:27.330324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:27.331013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:27.331295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:27.331569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:27.332309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:27.332653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:27.334015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:27.341650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 610/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:33:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:27.347780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:27.391548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:27.446720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a617c-4dd0-4f1c-993d-3d312b5034a5 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:27.478310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:27.481121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:27.481121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:27.481121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:27.490506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:27.491945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:27.509272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:27.510201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:27.511172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:27.511172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:27.511172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:27.511172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:27.511172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 614/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:27.516786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:27.613611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e871fa-e342-4397-80f7-cacb43bb3bb8 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:27.645978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:27.646636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:27.646922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:27.647227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:27.647945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:27.648254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:27.649687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:27.662585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 611/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:27.668510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:27.761924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:27.768490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d10aa-be1a-479e-84a8-1fd38159e780 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:27.801185 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:27.805887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:27.805887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:27.805887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:27.809743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:27.811172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:33:27.825699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:27.826697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:27.827752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 615/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:27.833035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:27.878737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:27.928739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af07b5-4480-46bd-bad8-22338043f6ce None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:27.960233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:27.960927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:27.961208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:27.961482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:27.962304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:27.962610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:27.964081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:27.981036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 612/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jul 20 17:33:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:27.990261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:28.088402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ff54-caf6-4b4e-b417-35767628bb77 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:28.123524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:28.124310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:28.124625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:28.124903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:28.135102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:28.136559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:28.153458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:28.158615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:28.160619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:28.160619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:28.160619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:28.160619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:28.160619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 616/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:28.161353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:28.258571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46654b-e99b-49ad-88c7-d498ac56e2bf None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:28.260850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:28.292988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:28.293651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:28.293938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:28.294215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:28.294987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:28.295292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:28.296698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:28.307455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 613/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:28.313122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:28.322515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:28.413259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95245-ba1f-4a3a-a2a2-a398407e5d92 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:28.445269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:28.446057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:28.446373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:28.446650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:28.453358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:28.454886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:33:28.470399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:28.472013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:28.473449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 617/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:28.478069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:28.575125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78917595-2392-4ffc-a90b-45114ca31727 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:28.607182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddad583-98b3-4c35-a274-9d3d1569f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:28.607832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddad583-98b3-4c35-a274-9d3d1569f50b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:28.608123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddad583-98b3-4c35-a274-9d3d1569f50b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:28.608399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddad583-98b3-4c35-a274-9d3d1569f50b None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:28.609209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddad583-98b3-4c35-a274-9d3d1569f50b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:28.609526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddad583-98b3-4c35-a274-9d3d1569f50b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:28.610956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddad583-98b3-4c35-a274-9d3d1569f50b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:28.622561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 614/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:33:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:28.629136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:28.699830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:28.732724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2356162-e692-4ca1-87a8-bfa3156fece3 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:28.764860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b000d05-ab86-4de1-ae9b-941bab7ad86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:28.765736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b000d05-ab86-4de1-ae9b-941bab7ad86f None admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:28.766026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b000d05-ab86-4de1-ae9b-941bab7ad86f None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:28.766301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b000d05-ab86-4de1-ae9b-941bab7ad86f None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/4871b13f87f1498ba33f8ecd2a6c0e39/roles/713299dfe855451c9fd4127a8cca33fc` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:28.780410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b000d05-ab86-4de1-ae9b-941bab7ad86f None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=4871b13f87f1498ba33f8ecd2a6c0e39, role_id=713299dfe855451c9fd4127a8cca33fc)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:28.781819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b000d05-ab86-4de1-ae9b-941bab7ad86f None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:28.885526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 618/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:33:28 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/4871b13f87f1498ba33f8ecd2a6c0e39/roles/713299dfe855451c9fd4127a8cca33fc => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:33:30.899812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d459f-65cc-4e55-b124-f2a32d3a11a4 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:30.900361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d459f-65cc-4e55-b124-f2a32d3a11a4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:30.900361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d459f-65cc-4e55-b124-f2a32d3a11a4 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:30.901774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 615/1233] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:30.906958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6365212-7901-4894-8de6-5920f6c70ead None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:30.907344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6365212-7901-4894-8de6-5920f6c70ead None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:30.907563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6365212-7901-4894-8de6-5920f6c70ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:30.923637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-a6365212-7901-4894-8de6-5920f6c70ead None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:30.925650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a6365212-7901-4894-8de6-5920f6c70ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:30.992369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 619/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:31.016233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d459f-65cc-4e55-b124-f2a32d3a11a4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:31.041586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:31.116501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d459f-65cc-4e55-b124-f2a32d3a11a4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:31.150500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b81274-5c49-4577-a24a-07725964948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:31.151254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b81274-5c49-4577-a24a-07725964948c None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:31.151534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b81274-5c49-4577-a24a-07725964948c None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:31.151806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b81274-5c49-4577-a24a-07725964948c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:31.161803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b81274-5c49-4577-a24a-07725964948c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-02b81274-5c49-4577-a24a-07725964948c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:33:31.169417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:31.170526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:31.171747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 616/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Jul 20 17:33:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:31.177116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6365212-7901-4894-8de6-5920f6c70ead None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:31.221251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:31.271471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6365212-7901-4894-8de6-5920f6c70ead None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:31.303395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:31.304055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:31.304342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:31.304654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:31.305396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:31.305704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:31.307060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:31.315088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 620/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Jul 20 17:33:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:31.321272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b81274-5c49-4577-a24a-07725964948c None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:31.402516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:31.417513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b81274-5c49-4577-a24a-07725964948c None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:31.449716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:31.450498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:31.450782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:31.451058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:31.460582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:31.462024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:31.479222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:31.480367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:31.488335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:31.488335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:31.488335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:31.488335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:31.488335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 617/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:31.488734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:31.582173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e49d4-7943-4931-b543-436b6f5941cf None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:31.591186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:31.615768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:31.616416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:31.616730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:31.617005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:31.617733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:31.618022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:31.619370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:31.630064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 621/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:31.635650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:31.730902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269b71f-116e-4894-a84f-675811c783dc None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:31.764891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:31.765691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:31.765978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:31.766257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:31.769531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:31.774063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:31.775517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:33:31.790570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:31.791782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:31.792972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 618/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:31.797541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:31.896133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1734332-9175-4eba-8e75-6f910f20ffec None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:31.929207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:31.929880 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:31.930166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:31.930447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:31.931289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:31.931604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:31.933077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:31.949816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 622/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jul 20 17:33:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:31.955597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:32.053164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed26301-b869-47cf-bea8-f5d2d50281d0 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:32.086399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:32.087043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:32.087330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:32.087614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:32.148160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:32.154566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:32.156020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:32.249667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 619/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:33:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:33:32.255981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:32.283492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:32.352195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6270e9f-d89f-4524-a31a-3d35f68f69c6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:32.383953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:32.384768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:32.385054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:32.385328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:32.395617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:32.397097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:32.414647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:32.421828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:32.422955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:32.422955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:32.422955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:32.422955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:32.422955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 623/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:32.423306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:32.511789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:32.517533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5457a4-93d4-4721-86d7-436c7bab8488 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:32.550768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:32.551423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:32.551702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:32.551972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:32.552711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:32.553015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:32.554367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:32.565094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 620/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:32.571302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:32.650080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:32.670134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b145cf-60a5-4cc4-8458-5752264c9bcd None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:32.709215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:32.710006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:32.710291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:32.710568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:32.716971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:32.718405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:33:32.731659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:32.732714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:32.733823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 624/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:33:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:32.739421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:32.837306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9670d-33dd-4ccc-8195-61e0d62ff182 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:32.869730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562dda2-63db-4568-bb63-7fcd9a47ceff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:32.870383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562dda2-63db-4568-bb63-7fcd9a47ceff None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:32.871139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562dda2-63db-4568-bb63-7fcd9a47ceff None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:32.871139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562dda2-63db-4568-bb63-7fcd9a47ceff None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:32.871694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562dda2-63db-4568-bb63-7fcd9a47ceff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:32.872006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562dda2-63db-4568-bb63-7fcd9a47ceff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:32.873471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562dda2-63db-4568-bb63-7fcd9a47ceff None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:32.885544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 621/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:32.891607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:32.989515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad09a-3e26-4d89-bd77-ff4f1aa5489d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:33.021437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95c24a-b8a2-4336-be74-bf16c6f25b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:33.022132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c24a-b8a2-4336-be74-bf16c6f25b12 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:33.022421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c24a-b8a2-4336-be74-bf16c6f25b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:33.022701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c24a-b8a2-4336-be74-bf16c6f25b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:33.023521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95c24a-b8a2-4336-be74-bf16c6f25b12 None admin] RBAC: Adding query filter params (role.id=713299dfe855451c9fd4127a8cca33fc, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=4871b13f87f1498ba33f8ecd2a6c0e39) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:33.023833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95c24a-b8a2-4336-be74-bf16c6f25b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:33.025277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95c24a-b8a2-4336-be74-bf16c6f25b12 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:33.030972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 625/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:33:32 2022] GET /identity/v3/role_assignments?user.id=4871b13f87f1498ba33f8ecd2a6c0e39&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=713299dfe855451c9fd4127a8cca33fc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:35.079528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db0ed7-ac6a-4353-a02e-b131f4e07a30 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:35.080082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db0ed7-ac6a-4353-a02e-b131f4e07a30 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:35.080082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db0ed7-ac6a-4353-a02e-b131f4e07a30 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:35.081989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 622/1247] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:35.087196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d236117-393d-4b14-a25c-d93cf17c1731 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:35.087614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d236117-393d-4b14-a25c-d93cf17c1731 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:35.087754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d236117-393d-4b14-a25c-d93cf17c1731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:35.092783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:35.105744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6d236117-393d-4b14-a25c-d93cf17c1731 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:35.106944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6d236117-393d-4b14-a25c-d93cf17c1731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:35.171086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 626/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:35.195142 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db0ed7-ac6a-4353-a02e-b131f4e07a30 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:35.213364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:35.293113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db0ed7-ac6a-4353-a02e-b131f4e07a30 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:35.325158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:35.326009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:35.326293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:35.326568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:35.336863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:33:35.344305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:35.345438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:35.346313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 623/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:33:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:35.351494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d236117-393d-4b14-a25c-d93cf17c1731 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:35.452460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d236117-393d-4b14-a25c-d93cf17c1731 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:35.482670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:35.488674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:35.489355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:35.489664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:35.490061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:35.490995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:35.491302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:35.492947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:35.502602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 627/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:33:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:35.512058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:35.589464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:35.611589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925feb64-9cc4-47d8-8d86-c8f46f74f382 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:35.642773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:35.644057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:35.644355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:35.644653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:35.654734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:35.656216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:35.673309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:35.679842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:35.680839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:35.680839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:35.680839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:35.680839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:35.680839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 624/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:35.681127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:35.778029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ef3b8-3a34-49c6-aa34-4fbe341289d7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:35.810826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:35.811476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:35.811759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:35.812031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:35.812770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:35.813074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:35.814475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:35.826435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 628/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:35.832182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:35.937973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be9241-2b1f-4f79-ac34-376f450d171f None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:35.963457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:35.970902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:35.972197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:35.972514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:35.972789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:35.979995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:35.981456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:33:35.995805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:35.996988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:35.998060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 625/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:36.003028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:36.058111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:36.102481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73150e0-3e10-41a9-841e-8adfc9c4a24d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:36.135501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:36.136164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:36.136469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:36.136753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:36.137588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:36.137895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:36.139313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:36.156035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 629/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jul 20 17:33:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:36.162210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:36.261658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5aaa-ef1b-444d-8027-567f8f719bee None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:36.295225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:36.295862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:36.296145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:36.296420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:36.350768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:36.363059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:36.364527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:36.457644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 626/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:33:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 17:33:36.463451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:36.560137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea4588-fd25-4475-92d2-99aebe502bf1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:36.574844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:36.595670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:36.596479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:36.597298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:36.597576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:36.607167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:36.608710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:36.625410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:36.626456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:36.627709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:36.627709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:36.627709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:36.627709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:36.627709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 630/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:36.632939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:36.719661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:36.730990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0eeb1d-5457-47bd-b13b-6f1a3d22a2de None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:36.763232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:36.763937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:36.764221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:36.764516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:36.765234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:36.765563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:36.767039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:36.778640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 627/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:36.784875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:36.882627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3e948-cf14-43cc-af02-dea500057b7e None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:36.915563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:36.916352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:36.916671 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:36.916949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:36.923501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:36.925033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:33:36.938481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:36.941027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:36.942252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 631/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:36.946068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:37.043649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6a50f-44a1-4abe-afd8-48ad9276a30e None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:37.077426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c46ec-3046-4dbd-8585-0cd8aa1497ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:37.081455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c46ec-3046-4dbd-8585-0cd8aa1497ec None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:37.081455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c46ec-3046-4dbd-8585-0cd8aa1497ec None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:37.081455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c46ec-3046-4dbd-8585-0cd8aa1497ec None admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:37.081455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c46ec-3046-4dbd-8585-0cd8aa1497ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:37.081455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c46ec-3046-4dbd-8585-0cd8aa1497ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:37.081455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c46ec-3046-4dbd-8585-0cd8aa1497ec None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:37.093182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 628/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:37.098912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:37.124700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:37.196377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b81b24-73e2-4dd7-b227-c518018c3707 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:37.229186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ce9b8-76e9-450f-98ed-ff42fca4b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:37.229831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce9b8-76e9-450f-98ed-ff42fca4b366 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:37.230123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce9b8-76e9-450f-98ed-ff42fca4b366 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:37.230405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce9b8-76e9-450f-98ed-ff42fca4b366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:37.231223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ce9b8-76e9-450f-98ed-ff42fca4b366 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=4871b13f87f1498ba33f8ecd2a6c0e39) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:37.231537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ce9b8-76e9-450f-98ed-ff42fca4b366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:37.233013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ce9b8-76e9-450f-98ed-ff42fca4b366 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:37.238745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 632/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:33:37 2022] GET /identity/v3/role_assignments?user.id=4871b13f87f1498ba33f8ecd2a6c0e39&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:33:39.274378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128efbf-ddc8-4b2d-8f03-805d02ca913e None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:39.274922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128efbf-ddc8-4b2d-8f03-805d02ca913e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:39.274922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128efbf-ddc8-4b2d-8f03-805d02ca913e None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:39.276317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 629/1261] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:39.281763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c709bf-9fab-48f5-8b71-f94e5b143e73 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:39.282199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c709bf-9fab-48f5-8b71-f94e5b143e73 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:39.282330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c709bf-9fab-48f5-8b71-f94e5b143e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:39.301681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-26c709bf-9fab-48f5-8b71-f94e5b143e73 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:39.302899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-26c709bf-9fab-48f5-8b71-f94e5b143e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:39.374907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 633/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:39.400281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128efbf-ddc8-4b2d-8f03-805d02ca913e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:39.413137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:39.514250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128efbf-ddc8-4b2d-8f03-805d02ca913e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:39.559340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:39.560134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:39.560424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:39.560731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:39.570976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:33:39.578577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:39.579500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:39.580462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 630/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:39.586358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c709bf-9fab-48f5-8b71-f94e5b143e73 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:39.605308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:39.682299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c709bf-9fab-48f5-8b71-f94e5b143e73 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:39.715385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:39.716037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:39.716318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:39.716619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:39.717356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:39.717658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:39.719003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:39.727365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 634/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:39.733261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:39.798196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:39.832600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656cc45-9d22-4393-9dc0-cd4da6430347 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:39.866639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:39.867421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:39.867999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:39.867999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:39.879407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:39.880892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:39.897870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:39.898951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:39.900174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:39.900174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:39.900174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:39.900174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:39.900174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 631/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:39.905394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:39.984183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:40.002763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169564e-4aac-457c-8c23-d04520c66728 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:40.035423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:40.036071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:40.036352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:40.036662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:40.037382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:40.037685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:40.039035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:40.049656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 635/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:40.056056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:40.158940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531da7c-b723-47f7-bf49-d5721718931d None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:40.181464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:40.195676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:40.196494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:40.196785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:40.197062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:40.204358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:40.205830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:33:40.219605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:40.220720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:40.221718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 632/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:40.226674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:40.324373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193b5a9-1249-4fc4-8c14-ffebf18007e7 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:40.356848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:40.357509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:40.357791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:40.358160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:40.359096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:40.359405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:40.360855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:40.370303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:40.378657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 636/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jul 20 17:33:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:40.384616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:40.482602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e9490-337f-4aa8-8e97-97d8de282068 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:40.515496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:40.516278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:40.516590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:40.516862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:40.526974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:40.528482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:40.545580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:40.546673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:40.547916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:40.547916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:40.547916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:40.547916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:40.547916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 633/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:40.553047 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:40.647672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27be18-b8b0-435b-bdbf-cfc567010ada None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:40.681220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:40.681866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:40.682146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:40.682419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:40.683130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:40.683438 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:40.684829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:40.695613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 637/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:40.701323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:40.735022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:40.804920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad5494-7a08-4fc6-bb56-cc3e8c20262b None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:40.838474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:40.839253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:40.839553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:40.839812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:40.846221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:40.848208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:33:40.861599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:40.862716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:40.863653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 634/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:33:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:40.868874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:40.927653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:40.966717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262e17-b91f-4364-9a92-541621eb793c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:40.999074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db1b24-e617-4f7c-90d6-b96919fb1a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:40.999785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db1b24-e617-4f7c-90d6-b96919fb1a70 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:41.000238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db1b24-e617-4f7c-90d6-b96919fb1a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:41.001018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db1b24-e617-4f7c-90d6-b96919fb1a70 None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:41.001808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db1b24-e617-4f7c-90d6-b96919fb1a70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:41.002120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db1b24-e617-4f7c-90d6-b96919fb1a70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:41.003567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db1b24-e617-4f7c-90d6-b96919fb1a70 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:41.015342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 638/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:33:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:41.022125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:41.112543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:41.120898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08640fc1-7da0-4290-8a20-6804ae23786a None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:41.154105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba1d0-2f08-422a-8513-8c5265d123e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:41.154981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba1d0-2f08-422a-8513-8c5265d123e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:41.155269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba1d0-2f08-422a-8513-8c5265d123e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:41.155549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba1d0-2f08-422a-8513-8c5265d123e9 None admin] PATH_INFO: `/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/4871b13f87f1498ba33f8ecd2a6c0e39/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:41.157541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ba1d0-2f08-422a-8513-8c5265d123e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4360ee3455064eea9e0b5197debd9a8f, user_id=4871b13f87f1498ba33f8ecd2a6c0e39, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:41.158981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ba1d0-2f08-422a-8513-8c5265d123e9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:41.294355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 635/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Jul 20 17:33:41 2022] PUT /identity/v3/projects/4360ee3455064eea9e0b5197debd9a8f/users/4871b13f87f1498ba33f8ecd2a6c0e39/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:33:43.298835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f558a-d3d5-4f3a-b1ad-0f2b99a5aac2 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:43.299571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f558a-d3d5-4f3a-b1ad-0f2b99a5aac2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:43.299571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f558a-d3d5-4f3a-b1ad-0f2b99a5aac2 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:43.301309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 639/1274] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:43.306587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441426a6-1144-4452-bf2f-71a5b4d659b5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:43.307015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441426a6-1144-4452-bf2f-71a5b4d659b5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:43.307198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441426a6-1144-4452-bf2f-71a5b4d659b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:43.324150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-441426a6-1144-4452-bf2f-71a5b4d659b5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:43.325366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-441426a6-1144-4452-bf2f-71a5b4d659b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:43.391814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 636/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:43.418020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f558a-d3d5-4f3a-b1ad-0f2b99a5aac2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:43.516507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f558a-d3d5-4f3a-b1ad-0f2b99a5aac2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:43.532939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:43.549598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:43.550904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:43.551186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:43.551479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:43.561862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 17:33:43.569480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:43.570635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:43.571534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 640/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Jul 20 17:33:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:43.576697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441426a6-1144-4452-bf2f-71a5b4d659b5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:43.654959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:43.673719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441426a6-1144-4452-bf2f-71a5b4d659b5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:43.706281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:43.707022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:43.707307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:43.707583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:43.708326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:43.708682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:43.710039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:43.718237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 637/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Jul 20 17:33:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:43.724697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:43.825032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8aedb-9443-4726-a482-db651efd9608 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:43.858144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:43.858922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:43.859206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:43.859479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:43.869980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:43.871424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:43.887944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:43.889183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:43.890474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:43.890474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:43.890474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:43.890474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:43.890474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 641/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 17:33:43.895451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:43.993095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e5fe8-751c-42bf-a562-1601655062cd None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:44.025562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:44.026218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:44.026539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:44.026820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:44.027530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:44.027836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:44.029239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:44.041314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 638/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:44.047021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:44.077376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:44.146236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2c7f0-86c4-46b7-b665-6e029411248d None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:44.180330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:44.181168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:44.181456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:44.181734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:44.189341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:44.191283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 17:33:44.205161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:44.207031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:44.207981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 642/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:44.212876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:44.216610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:44.311640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41741a37-ac70-41ee-a01b-89a13cd5cfac None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:44.345318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:44.345977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:44.346259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:44.346576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:44.347432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:44.347738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:44.349321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:44.365837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 639/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Jul 20 17:33:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:44.371666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:44.462408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:44.471621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4067e-9b7f-4cd2-a730-1a00fc7965a6 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:44.503740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:44.504403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:44.504720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:44.505045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:44.572486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:44.573983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:44.658278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:44.668847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 643/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Jul 20 17:33:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 17:33:44.674817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:44.734712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:44.772541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac631beb-0324-42b7-89be-0d91cbc57ef9 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:44.808278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:44.809090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:44.809373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:44.809647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:44.819299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:44.820869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 17:33:44.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:44.840129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:44.841119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 17:33:44.841119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 17:33:44.841119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 17:33:44.841119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:44.841119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 640/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:44.846674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:44.944755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71df426-f83b-4bd6-83e8-0ef8668b026c None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:44.977535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:44.978187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:44.978474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:44.978754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:44.979477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:44.979790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:44.981206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:44.992711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 644/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Jul 20 17:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:44.998457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:45.095802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f1fd2-f57d-4a26-ad53-e9d0d90be076 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:45.110564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:45.130340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:45.131133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:45.131419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:45.131697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:45.138231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:45.139678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 17:33:45.153580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:45.154537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: ERROR keystone.server.flask.application Jul 20 17:33:45.155521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 641/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Jul 20 17:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:45.161742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:45.211750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:45.260213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaad6f5-2d63-4919-9cc4-b540942046d1 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:45.292415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fcc71-6565-41b3-8e04-c0a4810ecacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:45.293091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fcc71-6565-41b3-8e04-c0a4810ecacc None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:45.293379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fcc71-6565-41b3-8e04-c0a4810ecacc None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:45.293657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fcc71-6565-41b3-8e04-c0a4810ecacc None admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:45.294437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fcc71-6565-41b3-8e04-c0a4810ecacc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:45.294749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fcc71-6565-41b3-8e04-c0a4810ecacc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:45.296233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fcc71-6565-41b3-8e04-c0a4810ecacc None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:45.308603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 645/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Jul 20 17:33:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:45.314759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:45.412751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0809e-5144-42a0-b060-f7717479de35 None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:45.445928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a91bd2-0577-4939-a18b-a5711d110d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:45.446570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a91bd2-0577-4939-a18b-a5711d110d43 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:45.446855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a91bd2-0577-4939-a18b-a5711d110d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:45.447133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a91bd2-0577-4939-a18b-a5711d110d43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:45.447953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a91bd2-0577-4939-a18b-a5711d110d43 None admin] RBAC: Adding query filter params (role.id=2f9c0373d6e7487d8f2f97663fe18ecf, scope.project.id=4360ee3455064eea9e0b5197debd9a8f, user.id=4871b13f87f1498ba33f8ecd2a6c0e39) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:45.448265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a91bd2-0577-4939-a18b-a5711d110d43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:45.449725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a91bd2-0577-4939-a18b-a5711d110d43 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:45.455401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 642/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Jul 20 17:33:45 2022] GET /identity/v3/role_assignments?user.id=4871b13f87f1498ba33f8ecd2a6c0e39&scope.project.id=4360ee3455064eea9e0b5197debd9a8f&role.id=2f9c0373d6e7487d8f2f97663fe18ecf => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:47.513399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eca6a-b0ed-497f-9533-d97e1b3f1ee9 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:47.513926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eca6a-b0ed-497f-9533-d97e1b3f1ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:47.513926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eca6a-b0ed-497f-9533-d97e1b3f1ee9 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:47.515363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 646/1288] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:47.521199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f63e5-c3d0-44cd-a855-7333cdaff7cc None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:47.521595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f63e5-c3d0-44cd-a855-7333cdaff7cc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:47.521753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f63e5-c3d0-44cd-a855-7333cdaff7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:47.538480 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-783f63e5-c3d0-44cd-a855-7333cdaff7cc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:47.539661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-783f63e5-c3d0-44cd-a855-7333cdaff7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:47.569528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:47.604865 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 643/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:47.627663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eca6a-b0ed-497f-9533-d97e1b3f1ee9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:47.729738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eca6a-b0ed-497f-9533-d97e1b3f1ee9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:47.740483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:47.762888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:47.763729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:47.763960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:47.764236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:47.764921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:47.766296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:47.777904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:47.779122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:47.780252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 647/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:33:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:47.785715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f63e5-c3d0-44cd-a855-7333cdaff7cc None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:47.882992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f63e5-c3d0-44cd-a855-7333cdaff7cc None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:47.915592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb74a1-d66a-4f2e-8711-4fba5023751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:47.916241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb74a1-d66a-4f2e-8711-4fba5023751b None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:47.916551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb74a1-d66a-4f2e-8711-4fba5023751b None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:47.916829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb74a1-d66a-4f2e-8711-4fba5023751b None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:47.917525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb74a1-d66a-4f2e-8711-4fba5023751b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:47.917942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb74a1-d66a-4f2e-8711-4fba5023751b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:47.919308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb74a1-d66a-4f2e-8711-4fba5023751b None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:47.923959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 644/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jul 20 17:33:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:47.930301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:48.028718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122f7a1-fc17-47a3-97d7-346b900c9d32 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:48.060335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a94e4-8e00-435c-a143-4551e231248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:48.061112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a94e4-8e00-435c-a143-4551e231248a None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:48.061398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a94e4-8e00-435c-a143-4551e231248a None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:48.061673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a94e4-8e00-435c-a143-4551e231248a None admin] PATH_INFO: `/v3/services` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:48.062368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a94e4-8e00-435c-a143-4551e231248a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:48.062672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a94e4-8e00-435c-a143-4551e231248a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:48.064033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a94e4-8e00-435c-a143-4551e231248a None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:48.072606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 648/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jul 20 17:33:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:50.079647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eb928-f392-4768-a2f1-7f86775cfd65 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:50.080171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eb928-f392-4768-a2f1-7f86775cfd65 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:50.080288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eb928-f392-4768-a2f1-7f86775cfd65 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:50.082315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 645/1293] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:50.087445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425556b4-6d4e-450f-8f37-5e0c3eb5e8ae None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:50.087904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425556b4-6d4e-450f-8f37-5e0c3eb5e8ae None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:50.088006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425556b4-6d4e-450f-8f37-5e0c3eb5e8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:50.105323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-425556b4-6d4e-450f-8f37-5e0c3eb5e8ae None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:50.109547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-425556b4-6d4e-450f-8f37-5e0c3eb5e8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:50.136672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:50.177083 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 649/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:50.200409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eb928-f392-4768-a2f1-7f86775cfd65 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:50.231651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:50.298119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eb928-f392-4768-a2f1-7f86775cfd65 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:50.330058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645d205-a2a6-44fa-b114-8ae4c90ebb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:50.330646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645d205-a2a6-44fa-b114-8ae4c90ebb95 None admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:50.330927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645d205-a2a6-44fa-b114-8ae4c90ebb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:50.331201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645d205-a2a6-44fa-b114-8ae4c90ebb95 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:50.332515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645d205-a2a6-44fa-b114-8ae4c90ebb95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:50.333867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645d205-a2a6-44fa-b114-8ae4c90ebb95 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:50.462902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 646/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Jul 20 17:33:50 2022] POST /identity/v3/services => generated 246 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:33:52.481541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cdf6b-1acb-43b0-99a1-b3b1bde45695 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:52.482086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cdf6b-1acb-43b0-99a1-b3b1bde45695 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:52.482086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cdf6b-1acb-43b0-99a1-b3b1bde45695 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:52.483483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 650/1296] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:52.488840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3344-3acd-4081-b1ca-f53eaeac0026 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:52.489225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3344-3acd-4081-b1ca-f53eaeac0026 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:52.489398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3344-3acd-4081-b1ca-f53eaeac0026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:52.506344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-dbeb3344-3acd-4081-b1ca-f53eaeac0026 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:52.507561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-dbeb3344-3acd-4081-b1ca-f53eaeac0026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:52.578566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 647/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:52.601638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cdf6b-1acb-43b0-99a1-b3b1bde45695 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:52.699437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cdf6b-1acb-43b0-99a1-b3b1bde45695 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:52.733089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:52.733817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:52.734102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:52.734376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:52.735049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:52.736411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:52.748585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:52.751568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:52.752651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 651/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:33:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:52.756819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb3344-3acd-4081-b1ca-f53eaeac0026 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:52.775066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:52.864454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb3344-3acd-4081-b1ca-f53eaeac0026 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:52.896654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:52.897312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:52.897590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:52.897862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:52.898556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:52.898859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:52.900213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:52.905282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 648/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jul 20 17:33:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:52.912216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:52.942436 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:53.007222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae325f0-9cb3-456b-9f01-61b062ddd8b2 None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:53.041149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606913d9-057c-4176-9be5-3fbb91c535d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:53.041793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606913d9-057c-4176-9be5-3fbb91c535d6 None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:53.042078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606913d9-057c-4176-9be5-3fbb91c535d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:53.042355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606913d9-057c-4176-9be5-3fbb91c535d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:53.043143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606913d9-057c-4176-9be5-3fbb91c535d6 None admin] RBAC: Adding query filter params (interface=public, service_id=9d297def8d3c454ca72b7d20fba8c5d5, region_id=RegionOne) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:53.043451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606913d9-057c-4176-9be5-3fbb91c535d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:53.044841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606913d9-057c-4176-9be5-3fbb91c535d6 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:53.049775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 652/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Jul 20 17:33:52 2022] GET /identity/v3/endpoints?service_id=9d297def8d3c454ca72b7d20fba8c5d5&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:53.055587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:53.137820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:53.155397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24808cf-85a3-4d0c-913c-9d3a1b080eeb None admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:53.188210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40facba1-4cdc-4b67-9cf8-817e65855702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:53.188897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40facba1-4cdc-4b67-9cf8-817e65855702 None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:53.189184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40facba1-4cdc-4b67-9cf8-817e65855702 None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:53.189463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40facba1-4cdc-4b67-9cf8-817e65855702 None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:53.193810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40facba1-4cdc-4b67-9cf8-817e65855702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:53.193810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40facba1-4cdc-4b67-9cf8-817e65855702 None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:53.198628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 649/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Jul 20 17:33:53 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:33:55.216691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5449de3-d086-498d-9b93-f20771aeaa9b None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:55.217234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5449de3-d086-498d-9b93-f20771aeaa9b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:55.217234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5449de3-d086-498d-9b93-f20771aeaa9b None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:55.218614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 653/1302] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:33:55.223916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9f67-fc88-413a-aa64-b6da851a2fed None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:55.224345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9f67-fc88-413a-aa64-b6da851a2fed None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:55.224513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9f67-fc88-413a-aa64-b6da851a2fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:55.241361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b7de9f67-fc88-413a-aa64-b6da851a2fed None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:33:55.242529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b7de9f67-fc88-413a-aa64-b6da851a2fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:33:55.307664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 650/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:33:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 17:33:55.330596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5449de3-d086-498d-9b93-f20771aeaa9b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:55.424674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:55.432118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5449de3-d086-498d-9b93-f20771aeaa9b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:55.464896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:55.465687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:55.465972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:55.466247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:55.466940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:55.468257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 17:33:55.480396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 17:33:55.481697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application with Lock( Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self._enter() Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application return self.creator() Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application created_value = creator( Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: ERROR keystone.server.flask.application Jul 20 17:33:55.482600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 654/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Jul 20 17:33:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 17:33:55.488079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de9f67-fc88-413a-aa64-b6da851a2fed None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:55.561619 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:33:55.584383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de9f67-fc88-413a-aa64-b6da851a2fed None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:55.617343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90ab98-8b05-4262-8c1b-10c2bdd15bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:55.617995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ab98-8b05-4262-8c1b-10c2bdd15bdc None admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:55.618277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ab98-8b05-4262-8c1b-10c2bdd15bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:55.618552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ab98-8b05-4262-8c1b-10c2bdd15bdc None admin] PATH_INFO: `/v3/services` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:55.619281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90ab98-8b05-4262-8c1b-10c2bdd15bdc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:33:55.619585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90ab98-8b05-4262-8c1b-10c2bdd15bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:55.621015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90ab98-8b05-4262-8c1b-10c2bdd15bdc None admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:55.625965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 651/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Jul 20 17:33:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:33:55.631861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:33:55.731143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064a247-6daf-4ff5-a342-75c806c5a6aa None admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:33:55.763269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679ddfc-e9ef-47be-93ea-6a8540673835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:33:55.763922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679ddfc-e9ef-47be-93ea-6a8540673835 None admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:33:55.764209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679ddfc-e9ef-47be-93ea-6a8540673835 None admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:33:55.764510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679ddfc-e9ef-47be-93ea-6a8540673835 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:33:55.765248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679ddfc-e9ef-47be-93ea-6a8540673835 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:33:55.766604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679ddfc-e9ef-47be-93ea-6a8540673835 None admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:33:55.879351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 655/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Jul 20 17:33:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:34:05.731282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=30 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:34:05.874711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:47.470058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f3a6-185e-4382-a23d-5e7668c35a16 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:47.470628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f3a6-185e-4382-a23d-5e7668c35a16 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:47.470628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0f3a6-185e-4382-a23d-5e7668c35a16 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:47.472127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 652/1307] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:40:47.477699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcdcdd-6b8c-4f64-9986-920b1e48c865 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:47.478636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcdcdd-6b8c-4f64-9986-920b1e48c865 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:47.479226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcdcdd-6b8c-4f64-9986-920b1e48c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:47.580037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-40fcdcdd-6b8c-4f64-9986-920b1e48c865 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:40:47.586517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-40fcdcdd-6b8c-4f64-9986-920b1e48c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:47.771971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 656/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:40:47.834069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa907c-c03f-497b-bae1-514c2b96a674 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:47.835453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa907c-c03f-497b-bae1-514c2b96a674 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:47.835612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa907c-c03f-497b-bae1-514c2b96a674 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:47.837591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 653/1309] 10.222.0.42 () {54 vars in 965 bytes} [Wed Jul 20 17:40:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:40:47.853419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578d9d2-9cc6-4a97-bd7b-f4a72600c85d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:47.853419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578d9d2-9cc6-4a97-bd7b-f4a72600c85d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:47.853419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578d9d2-9cc6-4a97-bd7b-f4a72600c85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:47.985464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c578d9d2-9cc6-4a97-bd7b-f4a72600c85d None None] MFA Rules not processed for user `d1d18a38f45e4501914e22689f145fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:40:47.986731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c578d9d2-9cc6-4a97-bd7b-f4a72600c85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:48.022505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:48.092128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 657/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Jul 20 17:40:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:40:48.109040 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa907c-c03f-497b-bae1-514c2b96a674 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:40:48.110831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8caa907c-c03f-497b-bae1-514c2b96a674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:48.252974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa907c-c03f-497b-bae1-514c2b96a674 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:40:48.335287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a95e7-5839-4c01-acf1-7ecb60c6ebb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:40:48.336370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a95e7-5839-4c01-acf1-7ecb60c6ebb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:48.336817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a95e7-5839-4c01-acf1-7ecb60c6ebb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:48.337095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a95e7-5839-4c01-acf1-7ecb60c6ebb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:48.338810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-015a95e7-5839-4c01-acf1-7ecb60c6ebb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:48.401213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:48.460423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a95e7-5839-4c01-acf1-7ecb60c6ebb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:40:48.462213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a95e7-5839-4c01-acf1-7ecb60c6ebb3 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:40:48.484483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 654/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:40:50.909060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47a2fa-d9dc-4f51-b05a-1d200191d008 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:50.909624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47a2fa-d9dc-4f51-b05a-1d200191d008 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:50.909624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47a2fa-d9dc-4f51-b05a-1d200191d008 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:50.911084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 658/1312] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:40:50.916354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159e90-92b2-439a-92f0-975d99ad2794 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:50.916853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159e90-92b2-439a-92f0-975d99ad2794 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:50.916955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159e90-92b2-439a-92f0-975d99ad2794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:50.985809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-07159e90-92b2-439a-92f0-975d99ad2794 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:40:50.986996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-07159e90-92b2-439a-92f0-975d99ad2794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:51.083812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 655/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:40:51.121833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47a2fa-d9dc-4f51-b05a-1d200191d008 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:40:51.123463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9b47a2fa-d9dc-4f51-b05a-1d200191d008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:51.230540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47a2fa-d9dc-4f51-b05a-1d200191d008 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:40:51.308846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:51.311644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eff589-820a-4c14-b4c3-d58368eec289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:40:51.312308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eff589-820a-4c14-b4c3-d58368eec289 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:51.312615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eff589-820a-4c14-b4c3-d58368eec289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:51.312892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eff589-820a-4c14-b4c3-d58368eec289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:51.314608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-42eff589-820a-4c14-b4c3-d58368eec289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:51.429200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eff589-820a-4c14-b4c3-d58368eec289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:40:51.430635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eff589-820a-4c14-b4c3-d58368eec289 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:40:51.448795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 659/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:40:54.138338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340dfba-7ea0-47f9-afa6-ef0a78821f4d None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:54.138823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340dfba-7ea0-47f9-afa6-ef0a78821f4d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:54.138915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340dfba-7ea0-47f9-afa6-ef0a78821f4d None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:54.140258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 656/1315] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:40:54.145430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1e75b-1647-4c2e-8819-b3a9b811acb1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:54.145830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1e75b-1647-4c2e-8819-b3a9b811acb1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:54.146019 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1e75b-1647-4c2e-8819-b3a9b811acb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:54.211393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-a0a1e75b-1647-4c2e-8819-b3a9b811acb1 None None] MFA Rules not processed for user `743bab5749aa46e6a980c8dede62afdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:40:54.212615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a0a1e75b-1647-4c2e-8819-b3a9b811acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:54.262867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:54.305676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 660/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:40:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:40:54.344790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340dfba-7ea0-47f9-afa6-ef0a78821f4d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:40:54.346309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0340dfba-7ea0-47f9-afa6-ef0a78821f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:54.355963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:54.451799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340dfba-7ea0-47f9-afa6-ef0a78821f4d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:40:54.522547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa34ad0-8034-429c-848c-6be1d09d071e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:40:54.523191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34ad0-8034-429c-848c-6be1d09d071e service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:54.523478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34ad0-8034-429c-848c-6be1d09d071e service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:54.523757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa34ad0-8034-429c-848c-6be1d09d071e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:54.525436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0fa34ad0-8034-429c-848c-6be1d09d071e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:54.586426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:54.631792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa34ad0-8034-429c-848c-6be1d09d071e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:40:54.633191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa34ad0-8034-429c-848c-6be1d09d071e service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:40:54.650936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 657/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:40:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:40:57.631908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc17b09-c49b-4f46-ac76-90731517266d None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:57.632294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc17b09-c49b-4f46-ac76-90731517266d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:57.632501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc17b09-c49b-4f46-ac76-90731517266d None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:57.633907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 661/1318] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:40:57.639046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db08e8-5f11-4e34-a717-22cf36257e27 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:57.639460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db08e8-5f11-4e34-a717-22cf36257e27 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:57.639598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db08e8-5f11-4e34-a717-22cf36257e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:57.702451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-76db08e8-5f11-4e34-a717-22cf36257e27 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:40:57.703635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-76db08e8-5f11-4e34-a717-22cf36257e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:57.798852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 658/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:40:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:40:57.851817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613030f-3840-46ae-b988-846e9637225e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:57.852102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613030f-3840-46ae-b988-846e9637225e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:57.852410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613030f-3840-46ae-b988-846e9637225e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:57.853837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 662/1320] 10.222.0.42 () {54 vars in 965 bytes} [Wed Jul 20 17:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:40:57.859541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e01f1f-a7c5-4454-8323-705071c68517 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:57.859836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e01f1f-a7c5-4454-8323-705071c68517 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:57.860145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e01f1f-a7c5-4454-8323-705071c68517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:57.881412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:57.934598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-85e01f1f-a7c5-4454-8323-705071c68517 None None] MFA Rules not processed for user `d1d18a38f45e4501914e22689f145fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:40:57.935865 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-85e01f1f-a7c5-4454-8323-705071c68517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:58.010374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 659/1321] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Jul 20 17:40:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:40:58.019367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613030f-3840-46ae-b988-846e9637225e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:40:58.020973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8613030f-3840-46ae-b988-846e9637225e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:58.121958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613030f-3840-46ae-b988-846e9637225e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:40:58.192479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e32e5a-dff1-4850-9abb-3ea1d45e15e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:40:58.193143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32e5a-dff1-4850-9abb-3ea1d45e15e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:40:58.193429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32e5a-dff1-4850-9abb-3ea1d45e15e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:40:58.193704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e32e5a-dff1-4850-9abb-3ea1d45e15e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:40:58.195348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b7e32e5a-dff1-4850-9abb-3ea1d45e15e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:40:58.203470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:40:58.301686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e32e5a-dff1-4850-9abb-3ea1d45e15e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:40:58.303143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e32e5a-dff1-4850-9abb-3ea1d45e15e2 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:40:58.321043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 663/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:40:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:02.099943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7672b-e593-4373-8230-9a904c3832b3 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:02.100337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7672b-e593-4373-8230-9a904c3832b3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:02.100551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7672b-e593-4373-8230-9a904c3832b3 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:02.102096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 660/1323] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:02.107309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824cd68-bd2e-4443-8099-87e2eb9c4b61 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:02.107773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824cd68-bd2e-4443-8099-87e2eb9c4b61 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:02.107883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824cd68-bd2e-4443-8099-87e2eb9c4b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:02.175764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e824cd68-bd2e-4443-8099-87e2eb9c4b61 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:02.177034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e824cd68-bd2e-4443-8099-87e2eb9c4b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:02.238256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:02.278766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 664/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:02.316109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7672b-e593-4373-8230-9a904c3832b3 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:02.317798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-fbc7672b-e593-4373-8230-9a904c3832b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:02.422989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7672b-e593-4373-8230-9a904c3832b3 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:02.436085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:02.499973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad0145-d987-413a-9bf6-0c4b53933a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:02.500657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad0145-d987-413a-9bf6-0c4b53933a4d service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:02.500945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad0145-d987-413a-9bf6-0c4b53933a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:02.501221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad0145-d987-413a-9bf6-0c4b53933a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:02.502879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5bad0145-d987-413a-9bf6-0c4b53933a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:02.616147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad0145-d987-413a-9bf6-0c4b53933a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:02.617577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad0145-d987-413a-9bf6-0c4b53933a4d service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:02.635653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 661/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:04.732641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe3b4-4e0b-49f0-aa8b-4aa9c9e424d2 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:04.733160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe3b4-4e0b-49f0-aa8b-4aa9c9e424d2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:04.733160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe3b4-4e0b-49f0-aa8b-4aa9c9e424d2 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:04.734642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 665/1326] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:04.740298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55eac6-9e46-4fee-82ce-794312c9cc65 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:04.740815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55eac6-9e46-4fee-82ce-794312c9cc65 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:04.740958 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55eac6-9e46-4fee-82ce-794312c9cc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:04.787565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:04.806195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-af55eac6-9e46-4fee-82ce-794312c9cc65 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:04.807430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-af55eac6-9e46-4fee-82ce-794312c9cc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:04.904067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 662/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:04.940801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe3b4-4e0b-49f0-aa8b-4aa9c9e424d2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:04.942330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-336fe3b4-4e0b-49f0-aa8b-4aa9c9e424d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:05.045242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe3b4-4e0b-49f0-aa8b-4aa9c9e424d2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:05.130220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc421b-24a1-46dd-87f8-371fe191fce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:05.130946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc421b-24a1-46dd-87f8-371fe191fce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:05.131227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc421b-24a1-46dd-87f8-371fe191fce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:05.131508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc421b-24a1-46dd-87f8-371fe191fce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:05.133342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-55cc421b-24a1-46dd-87f8-371fe191fce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:05.151410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:05.244315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc421b-24a1-46dd-87f8-371fe191fce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:05.245789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc421b-24a1-46dd-87f8-371fe191fce9 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:05.264251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 666/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:07.369983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045366b-97e1-4248-b4a8-56ed7e3cdf9b None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:07.370537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045366b-97e1-4248-b4a8-56ed7e3cdf9b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:07.370537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045366b-97e1-4248-b4a8-56ed7e3cdf9b None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:07.371892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 663/1329] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:07.377240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5837ff-dcc5-4e7c-814b-07c60064df4b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:07.377684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5837ff-dcc5-4e7c-814b-07c60064df4b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:07.377820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5837ff-dcc5-4e7c-814b-07c60064df4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:07.444700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3e5837ff-dcc5-4e7c-814b-07c60064df4b None None] MFA Rules not processed for user `743bab5749aa46e6a980c8dede62afdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:07.445945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3e5837ff-dcc5-4e7c-814b-07c60064df4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:07.511121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:07.545668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 667/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:07.584805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045366b-97e1-4248-b4a8-56ed7e3cdf9b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:07.586328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6045366b-97e1-4248-b4a8-56ed7e3cdf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:07.687613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045366b-97e1-4248-b4a8-56ed7e3cdf9b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:07.708345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:07.766521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f867e4-8835-4d5d-b53b-e352872ebf61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:07.767173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f867e4-8835-4d5d-b53b-e352872ebf61 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:07.767455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f867e4-8835-4d5d-b53b-e352872ebf61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:07.767788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f867e4-8835-4d5d-b53b-e352872ebf61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:07.769451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-36f867e4-8835-4d5d-b53b-e352872ebf61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:07.877394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f867e4-8835-4d5d-b53b-e352872ebf61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:07.878855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f867e4-8835-4d5d-b53b-e352872ebf61 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:07.902893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 664/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:10.473804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b9d0f-3710-4076-b192-55555caca545 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:10.474350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b9d0f-3710-4076-b192-55555caca545 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:10.474458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b9d0f-3710-4076-b192-55555caca545 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:10.475772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 668/1332] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:10.480940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd116d66-7c90-46b6-8e2e-007ab5772581 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:10.481358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd116d66-7c90-46b6-8e2e-007ab5772581 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:10.481515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd116d66-7c90-46b6-8e2e-007ab5772581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:10.542863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:10.545437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bd116d66-7c90-46b6-8e2e-007ab5772581 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:10.546645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bd116d66-7c90-46b6-8e2e-007ab5772581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:10.644786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 665/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:10.681242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b9d0f-3710-4076-b192-55555caca545 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:10.687610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7d2b9d0f-3710-4076-b192-55555caca545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:10.783229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b9d0f-3710-4076-b192-55555caca545 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:10.857393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ac3e0-74c0-44f4-a3bf-01958badd403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:10.858343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ac3e0-74c0-44f4-a3bf-01958badd403 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:10.858773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ac3e0-74c0-44f4-a3bf-01958badd403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:10.859190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ac3e0-74c0-44f4-a3bf-01958badd403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:10.861889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-429ac3e0-74c0-44f4-a3bf-01958badd403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:10.901423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:11.000447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ac3e0-74c0-44f4-a3bf-01958badd403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:11.001900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ac3e0-74c0-44f4-a3bf-01958badd403 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:11.019947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 669/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:13.101102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37e1d-e08b-45d8-bca0-6610321024b9 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:13.101556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37e1d-e08b-45d8-bca0-6610321024b9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:13.101662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f37e1d-e08b-45d8-bca0-6610321024b9 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:13.103031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 666/1335] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:13.108297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3f92-6368-418b-b142-90346cbebb2a None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:13.108759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3f92-6368-418b-b142-90346cbebb2a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:13.108892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3f92-6368-418b-b142-90346cbebb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:13.172700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c2de3f92-6368-418b-b142-90346cbebb2a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:13.173923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c2de3f92-6368-418b-b142-90346cbebb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:13.247335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:13.269472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 670/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:13.309889 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f37e1d-e08b-45d8-bca0-6610321024b9 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:13.311417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-21f37e1d-e08b-45d8-bca0-6610321024b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:13.412761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f37e1d-e08b-45d8-bca0-6610321024b9 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:13.441284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:13.485194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98b4aa-a47e-40d9-9265-285e8f3d3fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:13.485839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b4aa-a47e-40d9-9265-285e8f3d3fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:13.486125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b4aa-a47e-40d9-9265-285e8f3d3fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:13.486403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b4aa-a47e-40d9-9265-285e8f3d3fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:13.488022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-df98b4aa-a47e-40d9-9265-285e8f3d3fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:13.594056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98b4aa-a47e-40d9-9265-285e8f3d3fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:13.595483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98b4aa-a47e-40d9-9265-285e8f3d3fe8 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:13.613322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 667/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:16.337449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cf5e1-4cdb-423d-86bb-36daf3692baf None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:16.337970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cf5e1-4cdb-423d-86bb-36daf3692baf None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:16.338083 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cf5e1-4cdb-423d-86bb-36daf3692baf None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:16.339364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 671/1338] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:16.344566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1d9ad-3028-44d9-99f2-8a3db879007f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:16.344999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1d9ad-3028-44d9-99f2-8a3db879007f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:16.345103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1d9ad-3028-44d9-99f2-8a3db879007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:16.408801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c2f1d9ad-3028-44d9-99f2-8a3db879007f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:16.409923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c2f1d9ad-3028-44d9-99f2-8a3db879007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:16.418514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:16.505529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 668/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:16.541468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cf5e1-4cdb-423d-86bb-36daf3692baf None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:16.542993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e85cf5e1-4cdb-423d-86bb-36daf3692baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:16.646710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cf5e1-4cdb-423d-86bb-36daf3692baf None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:16.719617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42451600-c354-4ea4-a414-1ed42500f440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:16.720273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42451600-c354-4ea4-a414-1ed42500f440 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:16.720605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42451600-c354-4ea4-a414-1ed42500f440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:16.720886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42451600-c354-4ea4-a414-1ed42500f440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:16.722598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-42451600-c354-4ea4-a414-1ed42500f440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:16.757983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:16.830421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42451600-c354-4ea4-a414-1ed42500f440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:16.831862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42451600-c354-4ea4-a414-1ed42500f440 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:16.851115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 672/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:21.528541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65599892-b87b-4fad-a4f6-9f893d226084 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:21.534215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65599892-b87b-4fad-a4f6-9f893d226084 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:21.534215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65599892-b87b-4fad-a4f6-9f893d226084 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:21.534215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 669/1341] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:21.535824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a085c4-94d5-45bd-b493-bd0b9f7bd775 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:21.536232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a085c4-94d5-45bd-b493-bd0b9f7bd775 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:21.536411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a085c4-94d5-45bd-b493-bd0b9f7bd775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:21.608956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d3a085c4-94d5-45bd-b493-bd0b9f7bd775 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:21.610515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d3a085c4-94d5-45bd-b493-bd0b9f7bd775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:21.706863 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:21.722238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 673/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:41:21.761890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65599892-b87b-4fad-a4f6-9f893d226084 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:21.763468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-65599892-b87b-4fad-a4f6-9f893d226084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:21.865604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65599892-b87b-4fad-a4f6-9f893d226084 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:21.904175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:21.939435 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039e57f-1aca-4fd1-87bf-b2094820a53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:21.940084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039e57f-1aca-4fd1-87bf-b2094820a53a service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:21.940423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039e57f-1aca-4fd1-87bf-b2094820a53a service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:21.940931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039e57f-1aca-4fd1-87bf-b2094820a53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:21.944889 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a039e57f-1aca-4fd1-87bf-b2094820a53a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:22.053286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039e57f-1aca-4fd1-87bf-b2094820a53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:22.054626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039e57f-1aca-4fd1-87bf-b2094820a53a service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:22.073406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 670/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:25.213783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5b690-03a5-4900-a701-405e3da6091e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:25.214407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5b690-03a5-4900-a701-405e3da6091e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:25.214407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5b690-03a5-4900-a701-405e3da6091e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:25.215876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 674/1344] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:25.220980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936c0aa-c848-4af5-94fd-6b85916be67c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:25.221381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936c0aa-c848-4af5-94fd-6b85916be67c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:25.221657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936c0aa-c848-4af5-94fd-6b85916be67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:25.285395 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b936c0aa-c848-4af5-94fd-6b85916be67c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:25.287186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b936c0aa-c848-4af5-94fd-6b85916be67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:25.305080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:25.383560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 671/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:25.421127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5b690-03a5-4900-a701-405e3da6091e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:25.422685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-60a5b690-03a5-4900-a701-405e3da6091e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:25.528113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5b690-03a5-4900-a701-405e3da6091e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:25.604668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21604f14-094e-4cd1-9749-ab90308b5f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:25.605322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21604f14-094e-4cd1-9749-ab90308b5f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:25.605609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21604f14-094e-4cd1-9749-ab90308b5f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:25.605888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21604f14-094e-4cd1-9749-ab90308b5f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:25.607569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-21604f14-094e-4cd1-9749-ab90308b5f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:25.653446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:25.718042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21604f14-094e-4cd1-9749-ab90308b5f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:25.719790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21604f14-094e-4cd1-9749-ab90308b5f64 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:25.738907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 675/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:30.080326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3a7aa-4862-4ee8-860f-ee21d7e19d39 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:30.080849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3a7aa-4862-4ee8-860f-ee21d7e19d39 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:30.080947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3a7aa-4862-4ee8-860f-ee21d7e19d39 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:30.082310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 672/1347] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:30.087955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e0381-f663-4cd7-bc81-1d8f877a8aea None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:30.098648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e0381-f663-4cd7-bc81-1d8f877a8aea None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:30.098648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e0381-f663-4cd7-bc81-1d8f877a8aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:30.159431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-207e0381-f663-4cd7-bc81-1d8f877a8aea None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:30.160620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-207e0381-f663-4cd7-bc81-1d8f877a8aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:30.257946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:30.260522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 676/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:30.299392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3a7aa-4862-4ee8-860f-ee21d7e19d39 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:30.301283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b3f3a7aa-4862-4ee8-860f-ee21d7e19d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:30.408736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3a7aa-4862-4ee8-860f-ee21d7e19d39 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:30.457821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:30.484875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3aa15-0812-435e-bf88-e093cf6c7f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:30.485524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3aa15-0812-435e-bf88-e093cf6c7f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:30.485815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3aa15-0812-435e-bf88-e093cf6c7f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:30.486092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3aa15-0812-435e-bf88-e093cf6c7f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:30.487722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ffb3aa15-0812-435e-bf88-e093cf6c7f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:30.595421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3aa15-0812-435e-bf88-e093cf6c7f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:30.596903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3aa15-0812-435e-bf88-e093cf6c7f72 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:30.615004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 673/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:35.513377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc453b-0be2-43be-91bb-594a81d0b804 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:35.513885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc453b-0be2-43be-91bb-594a81d0b804 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:35.514227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc453b-0be2-43be-91bb-594a81d0b804 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:35.516480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 677/1350] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:41:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:35.526302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b698e90-660d-4d02-9711-9fed03e2ff05 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:35.526302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b698e90-660d-4d02-9711-9fed03e2ff05 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:35.526302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b698e90-660d-4d02-9711-9fed03e2ff05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:35.645882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3b698e90-660d-4d02-9711-9fed03e2ff05 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:35.647249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3b698e90-660d-4d02-9711-9fed03e2ff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:35.669507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:35.768368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 674/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:41:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:40.088597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630af233-d1e1-431f-8b98-b2f290255f0e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:40.089195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630af233-d1e1-431f-8b98-b2f290255f0e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:40.089195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630af233-d1e1-431f-8b98-b2f290255f0e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:40.090678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 678/1352] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:40.100418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67671852-681d-4763-a094-0083b6539523 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:40.101396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67671852-681d-4763-a094-0083b6539523 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:40.105303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67671852-681d-4763-a094-0083b6539523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:40.197828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-67671852-681d-4763-a094-0083b6539523 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:40.199510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-67671852-681d-4763-a094-0083b6539523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:40.282115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:40.322770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 675/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:41.462430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cd037-aae5-4c17-8e13-d21458dec20f None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:41.462980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cd037-aae5-4c17-8e13-d21458dec20f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:41.462980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cd037-aae5-4c17-8e13-d21458dec20f None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:41.464496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 679/1354] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:41.469528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b83f4-8f3f-49ab-941a-c3590e1925c5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:41.469967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b83f4-8f3f-49ab-941a-c3590e1925c5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:41.470118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b83f4-8f3f-49ab-941a-c3590e1925c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:41.545057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8a1b83f4-8f3f-49ab-941a-c3590e1925c5 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:41.546254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8a1b83f4-8f3f-49ab-941a-c3590e1925c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:41.651467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 676/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:45.107750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cd9d8-163e-4619-a3a7-05ca32d40df0 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:45.108245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cd9d8-163e-4619-a3a7-05ca32d40df0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:45.110565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cd9d8-163e-4619-a3a7-05ca32d40df0 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:45.122961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 680/1356] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:45 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:45.128825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a11442-33f4-4c1b-90ee-1fce78daec22 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:45.129279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a11442-33f4-4c1b-90ee-1fce78daec22 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:45.129390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a11442-33f4-4c1b-90ee-1fce78daec22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:45.170320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:45.203996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-86a11442-33f4-4c1b-90ee-1fce78daec22 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:45.205270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-86a11442-33f4-4c1b-90ee-1fce78daec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:45.316579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 677/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:45.363516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c4bb9-de1e-4a41-988f-a1c6d0453bcf None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:45.364224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c4bb9-de1e-4a41-988f-a1c6d0453bcf None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:45.365024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c4bb9-de1e-4a41-988f-a1c6d0453bcf None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:45.368207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 681/1358] 10.222.0.42 () {54 vars in 962 bytes} [Wed Jul 20 17:41:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:45.375396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db681a9-8cf6-4af2-964c-46996ff68a78 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:45.375687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db681a9-8cf6-4af2-964c-46996ff68a78 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:45.375965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db681a9-8cf6-4af2-964c-46996ff68a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:45.457319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1db681a9-8cf6-4af2-964c-46996ff68a78 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:45.458564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1db681a9-8cf6-4af2-964c-46996ff68a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:45.509567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:45.578910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 678/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Jul 20 17:41:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:45.586847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c4bb9-de1e-4a41-988f-a1c6d0453bcf None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:45.588578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d22c4bb9-de1e-4a41-988f-a1c6d0453bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:45.737677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c4bb9-de1e-4a41-988f-a1c6d0453bcf None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:45.811653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:45.851308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450e39a-c2fb-415f-8d69-1f6648199ef4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:45.851953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450e39a-c2fb-415f-8d69-1f6648199ef4 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:45.852233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450e39a-c2fb-415f-8d69-1f6648199ef4 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:45.852535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450e39a-c2fb-415f-8d69-1f6648199ef4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:45.854144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8450e39a-c2fb-415f-8d69-1f6648199ef4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:45.947987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450e39a-c2fb-415f-8d69-1f6648199ef4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:45.949618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450e39a-c2fb-415f-8d69-1f6648199ef4 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:45.969081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 682/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:48.272715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67484d-ac95-4a82-a49f-91ab0b9b8b27 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:48.272715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67484d-ac95-4a82-a49f-91ab0b9b8b27 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:48.273236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67484d-ac95-4a82-a49f-91ab0b9b8b27 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:48.275427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 679/1361] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:48.280233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52443672-87bd-42da-9dcb-31fa0223041e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:48.280666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52443672-87bd-42da-9dcb-31fa0223041e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:48.281407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52443672-87bd-42da-9dcb-31fa0223041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:48.352390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-52443672-87bd-42da-9dcb-31fa0223041e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:48.353624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-52443672-87bd-42da-9dcb-31fa0223041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:48.399521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:48.469041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 683/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:48.517698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67484d-ac95-4a82-a49f-91ab0b9b8b27 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:48.519372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2f67484d-ac95-4a82-a49f-91ab0b9b8b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:48.693762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67484d-ac95-4a82-a49f-91ab0b9b8b27 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:48.704283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:48.823032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5a584-9f88-44b6-9ce0-4d1a47f084a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:48.823696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5a584-9f88-44b6-9ce0-4d1a47f084a8 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:48.823982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5a584-9f88-44b6-9ce0-4d1a47f084a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:48.824262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5a584-9f88-44b6-9ce0-4d1a47f084a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:48.825986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-90f5a584-9f88-44b6-9ce0-4d1a47f084a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:48.921705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5a584-9f88-44b6-9ce0-4d1a47f084a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:48.928069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5a584-9f88-44b6-9ce0-4d1a47f084a8 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:48.942603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 680/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:51.579046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999d67-842b-49fb-b958-569109b06844 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:51.579637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999d67-842b-49fb-b958-569109b06844 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:51.579750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999d67-842b-49fb-b958-569109b06844 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:51.581087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 684/1364] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:51.587184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68652aa-f9c5-4319-8d49-e6b88900c6e4 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:51.587589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68652aa-f9c5-4319-8d49-e6b88900c6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:51.587778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68652aa-f9c5-4319-8d49-e6b88900c6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:51.602601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:51.657543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a68652aa-f9c5-4319-8d49-e6b88900c6e4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:51.658753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a68652aa-f9c5-4319-8d49-e6b88900c6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:51.760485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 681/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:41:51.804740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59654e02-6a8f-4fe9-b452-3de24395e8ba None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:51.805234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59654e02-6a8f-4fe9-b452-3de24395e8ba None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:51.805618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59654e02-6a8f-4fe9-b452-3de24395e8ba None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:51.807165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 685/1366] 10.222.0.42 () {54 vars in 962 bytes} [Wed Jul 20 17:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:51.812581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e2d22-2ff9-4412-aef2-e4df80a7f685 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:51.812874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e2d22-2ff9-4412-aef2-e4df80a7f685 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:51.813154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e2d22-2ff9-4412-aef2-e4df80a7f685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:51.893249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4a7e2d22-2ff9-4412-aef2-e4df80a7f685 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:51.894552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4a7e2d22-2ff9-4412-aef2-e4df80a7f685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:51.941306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:52.021725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 682/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Jul 20 17:41:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:52.029616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59654e02-6a8f-4fe9-b452-3de24395e8ba None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:52.031916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-59654e02-6a8f-4fe9-b452-3de24395e8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:52.192581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59654e02-6a8f-4fe9-b452-3de24395e8ba None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:52.213930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:52.309324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b426a-4c7c-44d8-925a-c425a93a91a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:52.310153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b426a-4c7c-44d8-925a-c425a93a91a9 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:52.310545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b426a-4c7c-44d8-925a-c425a93a91a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:52.310928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b426a-4c7c-44d8-925a-c425a93a91a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:52.312710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-718b426a-4c7c-44d8-925a-c425a93a91a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:52.407982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b426a-4c7c-44d8-925a-c425a93a91a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:52.409802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b426a-4c7c-44d8-925a-c425a93a91a9 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:52.433359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 686/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:53.350185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cada96-e687-4a53-ad1a-a8bb7fef5688 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:53.350710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cada96-e687-4a53-ad1a-a8bb7fef5688 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:53.350973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cada96-e687-4a53-ad1a-a8bb7fef5688 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:53.352665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 683/1369] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:53.357810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db859d-dc6f-4240-9974-3435d035f72e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:53.358252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db859d-dc6f-4240-9974-3435d035f72e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:53.358348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db859d-dc6f-4240-9974-3435d035f72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:53.390587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:53.429865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-00db859d-dc6f-4240-9974-3435d035f72e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:53.431075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-00db859d-dc6f-4240-9974-3435d035f72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:53.536470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 687/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:53.877002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9a312-b146-4738-97b6-7df30d9f23b0 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:53.878316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9a312-b146-4738-97b6-7df30d9f23b0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:53.878316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9a312-b146-4738-97b6-7df30d9f23b0 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:53.879129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 684/1371] 10.222.0.42 () {54 vars in 964 bytes} [Wed Jul 20 17:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:53.884258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea587-bdd1-40fd-bff4-f3a4d43d23a0 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:53.884596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea587-bdd1-40fd-bff4-f3a4d43d23a0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:53.884885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbea587-bdd1-40fd-bff4-f3a4d43d23a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:53.966582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-fcbea587-bdd1-40fd-bff4-f3a4d43d23a0 None None] MFA Rules not processed for user `9706ef808a224c83aca165c595b596ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:53.968252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fcbea587-bdd1-40fd-bff4-f3a4d43d23a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:54.019246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:54.051106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 688/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Jul 20 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:54.059476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9a312-b146-4738-97b6-7df30d9f23b0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:54.061166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6fb9a312-b146-4738-97b6-7df30d9f23b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:54.187029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9a312-b146-4738-97b6-7df30d9f23b0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:54.229243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:54.263241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca530aa-952f-4637-a958-2d5b6a68dcd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:54.263901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca530aa-952f-4637-a958-2d5b6a68dcd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:54.264189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca530aa-952f-4637-a958-2d5b6a68dcd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:54.264493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca530aa-952f-4637-a958-2d5b6a68dcd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:54.266185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7ca530aa-952f-4637-a958-2d5b6a68dcd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:54.386776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca530aa-952f-4637-a958-2d5b6a68dcd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:54.388240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca530aa-952f-4637-a958-2d5b6a68dcd7 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:54.407810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 685/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:41:54.801646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca8138-c1a9-452a-9d47-da0320054685 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:54.802075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca8138-c1a9-452a-9d47-da0320054685 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:54.802414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca8138-c1a9-452a-9d47-da0320054685 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:54.803640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 689/1374] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:54.809463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5d0a-d88f-48ba-997f-65febe6d500b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:54.810171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5d0a-d88f-48ba-997f-65febe6d500b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:54.810171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5d0a-d88f-48ba-997f-65febe6d500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:54.894319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-97cb5d0a-d88f-48ba-997f-65febe6d500b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:54.895530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-97cb5d0a-d88f-48ba-997f-65febe6d500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:54.916105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:54.991926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 686/1375] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:55.037135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca8138-c1a9-452a-9d47-da0320054685 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:55.038689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-44ca8138-c1a9-452a-9d47-da0320054685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:55.203975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca8138-c1a9-452a-9d47-da0320054685 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:55.247419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:55.302963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a59b3-6dbd-4264-9577-e354b02efd8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:55.304232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a59b3-6dbd-4264-9577-e354b02efd8f service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:55.304560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a59b3-6dbd-4264-9577-e354b02efd8f service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:55.304835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a59b3-6dbd-4264-9577-e354b02efd8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:55.306475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b71a59b3-6dbd-4264-9577-e354b02efd8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:55.394140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a59b3-6dbd-4264-9577-e354b02efd8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:55.395845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a59b3-6dbd-4264-9577-e354b02efd8f service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:55.413959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 690/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:41:57.640854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63d1e-eb71-4cf6-a4d4-1af38fb1a76c None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:57.641366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63d1e-eb71-4cf6-a4d4-1af38fb1a76c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:57.641366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63d1e-eb71-4cf6-a4d4-1af38fb1a76c None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:57.642740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 687/1377] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:41:57.648525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb7658-b154-4dd7-942e-29748075b3bf None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:57.648928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb7658-b154-4dd7-942e-29748075b3bf None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:57.649091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb7658-b154-4dd7-942e-29748075b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:57.714350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-06cb7658-b154-4dd7-942e-29748075b3bf None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:41:57.715544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-06cb7658-b154-4dd7-942e-29748075b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:57.721953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:57.818119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 691/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:41:57.861920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63d1e-eb71-4cf6-a4d4-1af38fb1a76c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:41:57.863430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-31a63d1e-eb71-4cf6-a4d4-1af38fb1a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:58.043884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63d1e-eb71-4cf6-a4d4-1af38fb1a76c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:41:58.048776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:41:58.200051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162ce7b-9896-48de-bacb-6a4021a5792e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:41:58.201066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162ce7b-9896-48de-bacb-6a4021a5792e service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:41:58.201622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162ce7b-9896-48de-bacb-6a4021a5792e service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:41:58.202069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162ce7b-9896-48de-bacb-6a4021a5792e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:41:58.203878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c162ce7b-9896-48de-bacb-6a4021a5792e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:41:58.300028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162ce7b-9896-48de-bacb-6a4021a5792e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:41:58.301986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162ce7b-9896-48de-bacb-6a4021a5792e service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:41:58.321981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 688/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:00.573990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed58ac-fa56-4036-9488-ee39176e776b None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:00.574533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed58ac-fa56-4036-9488-ee39176e776b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:00.574533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed58ac-fa56-4036-9488-ee39176e776b None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:00.576091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 692/1380] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:00.581422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f2dbc-5a16-4639-a32f-6d37c00882cc None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:00.581850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f2dbc-5a16-4639-a32f-6d37c00882cc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:00.582029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f2dbc-5a16-4639-a32f-6d37c00882cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:00.592235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:00.647414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d44f2dbc-5a16-4639-a32f-6d37c00882cc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:00.648650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d44f2dbc-5a16-4639-a32f-6d37c00882cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:00.751373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 689/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:00.795792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed58ac-fa56-4036-9488-ee39176e776b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:00.797447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-eaed58ac-fa56-4036-9488-ee39176e776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:00.967140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:00.974084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed58ac-fa56-4036-9488-ee39176e776b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:01.093241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335538c2-03fa-44ca-897e-40be554ab347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:01.093932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335538c2-03fa-44ca-897e-40be554ab347 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:01.094225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335538c2-03fa-44ca-897e-40be554ab347 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:01.094509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335538c2-03fa-44ca-897e-40be554ab347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:01.096217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-335538c2-03fa-44ca-897e-40be554ab347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:01.193688 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335538c2-03fa-44ca-897e-40be554ab347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:01.195294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335538c2-03fa-44ca-897e-40be554ab347 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:01.206709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:01.215605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 693/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:03.605872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901e8a-8686-4924-b4c2-ca68aab73edd None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:03.606436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901e8a-8686-4924-b4c2-ca68aab73edd None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:03.606436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901e8a-8686-4924-b4c2-ca68aab73edd None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:03.607905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 690/1383] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:03.614503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798508cd-e9f8-44cd-8b8b-c443185e7149 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:03.614904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798508cd-e9f8-44cd-8b8b-c443185e7149 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:03.615297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798508cd-e9f8-44cd-8b8b-c443185e7149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:03.699683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-798508cd-e9f8-44cd-8b8b-c443185e7149 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:03.701296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-798508cd-e9f8-44cd-8b8b-c443185e7149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:03.802844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 694/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:03.846048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af901e8a-8686-4924-b4c2-ca68aab73edd None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:03.847554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-af901e8a-8686-4924-b4c2-ca68aab73edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:03.932863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:03.997155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af901e8a-8686-4924-b4c2-ca68aab73edd None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:04.124026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9b073-421c-44cf-8ee4-2a726cdeb168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:04.124862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9b073-421c-44cf-8ee4-2a726cdeb168 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:04.125211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9b073-421c-44cf-8ee4-2a726cdeb168 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:04.125555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9b073-421c-44cf-8ee4-2a726cdeb168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:04.127700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-41d9b073-421c-44cf-8ee4-2a726cdeb168 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:04.192913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:04.233437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9b073-421c-44cf-8ee4-2a726cdeb168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:04.235529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9b073-421c-44cf-8ee4-2a726cdeb168 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:04.258833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 691/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:42:06.370992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486590c-68d0-407c-834a-4992a2c5815c None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:06.371527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486590c-68d0-407c-834a-4992a2c5815c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:06.371527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486590c-68d0-407c-834a-4992a2c5815c None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:06.372976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 695/1386] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:06.378793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92a07-bd7e-4656-8264-aa838f483830 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:06.379252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92a07-bd7e-4656-8264-aa838f483830 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:06.379529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92a07-bd7e-4656-8264-aa838f483830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:06.462235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-38e92a07-bd7e-4656-8264-aa838f483830 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:06.463774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-38e92a07-bd7e-4656-8264-aa838f483830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:06.571836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 692/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:06.624376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486590c-68d0-407c-834a-4992a2c5815c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:06.625802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3486590c-68d0-407c-834a-4992a2c5815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:06.708881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:06.782124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486590c-68d0-407c-834a-4992a2c5815c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:06.898170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358430d1-de32-4666-86ac-cdac91f1342d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:06.898744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358430d1-de32-4666-86ac-cdac91f1342d service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:06.899028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358430d1-de32-4666-86ac-cdac91f1342d service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:06.899306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358430d1-de32-4666-86ac-cdac91f1342d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:06.901015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-358430d1-de32-4666-86ac-cdac91f1342d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:06.940393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:06.992144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358430d1-de32-4666-86ac-cdac91f1342d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:06.993936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358430d1-de32-4666-86ac-cdac91f1342d service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:07.013565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 696/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:09.262110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc30541-b66f-4c35-a75e-b12abad256b7 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:09.262697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc30541-b66f-4c35-a75e-b12abad256b7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:09.262697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc30541-b66f-4c35-a75e-b12abad256b7 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:09.264114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 693/1389] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:09.269424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e3a1f-eea8-4151-8ab5-93d3fac8d571 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:09.269872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e3a1f-eea8-4151-8ab5-93d3fac8d571 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:09.269978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e3a1f-eea8-4151-8ab5-93d3fac8d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:09.333422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-978e3a1f-eea8-4151-8ab5-93d3fac8d571 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:09.335161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-978e3a1f-eea8-4151-8ab5-93d3fac8d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:09.431494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 697/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:09.475713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc30541-b66f-4c35-a75e-b12abad256b7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:09.477756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-afc30541-b66f-4c35-a75e-b12abad256b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:09.508974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:09.624982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc30541-b66f-4c35-a75e-b12abad256b7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:09.729181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:09.740400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0a36-a33d-444f-bd76-c651ec836ece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:09.741104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0a36-a33d-444f-bd76-c651ec836ece service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:09.741390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0a36-a33d-444f-bd76-c651ec836ece service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:09.741666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0a36-a33d-444f-bd76-c651ec836ece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:09.743319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-879a0a36-a33d-444f-bd76-c651ec836ece service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:09.843678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a0a36-a33d-444f-bd76-c651ec836ece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:09.845303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a0a36-a33d-444f-bd76-c651ec836ece service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:09.865351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 694/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:09.920992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eafaf8-fa67-447a-ba0e-6fe1acfc3d7a None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:09.921283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eafaf8-fa67-447a-ba0e-6fe1acfc3d7a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:09.921565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eafaf8-fa67-447a-ba0e-6fe1acfc3d7a None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:09.923539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 698/1392] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:09.929177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fc1e6-db7b-4705-93a0-8ef740dab982 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:09.929465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fc1e6-db7b-4705-93a0-8ef740dab982 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:09.929738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fc1e6-db7b-4705-93a0-8ef740dab982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:09.999050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-749fc1e6-db7b-4705-93a0-8ef740dab982 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:10.000290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-749fc1e6-db7b-4705-93a0-8ef740dab982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:10.059370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:10.097054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 695/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:10.152494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f0497-fea8-406b-95b5-3f682959285a None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:10.152792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f0497-fea8-406b-95b5-3f682959285a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:10.153076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f0497-fea8-406b-95b5-3f682959285a None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:10.154504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 699/1394] 10.222.0.42 () {54 vars in 966 bytes} [Wed Jul 20 17:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:10.159358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c45e2-49ca-4f92-9432-c999d6c90302 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:10.159650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c45e2-49ca-4f92-9432-c999d6c90302 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:10.159927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c45e2-49ca-4f92-9432-c999d6c90302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:10.232063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-441c45e2-49ca-4f92-9432-c999d6c90302 None None] MFA Rules not processed for user `de1a294634524a9d8ff48c7c50726228`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:10.233292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-441c45e2-49ca-4f92-9432-c999d6c90302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:10.316934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 696/1395] 10.222.0.42 () {58 vars in 1063 bytes} [Wed Jul 20 17:42:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:10.324565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f0497-fea8-406b-95b5-3f682959285a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:10.326192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-760f0497-fea8-406b-95b5-3f682959285a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:10.340672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:10.434415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f0497-fea8-406b-95b5-3f682959285a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:10.506737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa52b9-8926-47d5-a848-f5361c846967 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:10.507384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa52b9-8926-47d5-a848-f5361c846967 service glance] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:10.507667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa52b9-8926-47d5-a848-f5361c846967 service glance] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:10.507943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa52b9-8926-47d5-a848-f5361c846967 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:10.509619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-65fa52b9-8926-47d5-a848-f5361c846967 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:10.576325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:10.615322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa52b9-8926-47d5-a848-f5361c846967 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:10.617722 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa52b9-8926-47d5-a848-f5361c846967 service glance] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:10.635890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 700/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:42:11.517507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb20425-f6cd-4af4-b6af-e1f33df4091b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:11.517967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb20425-f6cd-4af4-b6af-e1f33df4091b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:11.518139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb20425-f6cd-4af4-b6af-e1f33df4091b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:11.584799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-dbb20425-f6cd-4af4-b6af-e1f33df4091b None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:11.586033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-dbb20425-f6cd-4af4-b6af-e1f33df4091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:11.594199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:11.680879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 697/1397] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:42:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:11.729270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1fab-3fa8-42db-a746-33d64b84bb5e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:11.729715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1fab-3fa8-42db-a746-33d64b84bb5e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:11.729820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1fab-3fa8-42db-a746-33d64b84bb5e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:11.731258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 701/1398] 10.222.0.42 () {54 vars in 943 bytes} [Wed Jul 20 17:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:11.737452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347c83b-1b5a-4d64-896a-8d1f68bef575 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:11.737743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347c83b-1b5a-4d64-896a-8d1f68bef575 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:11.738022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347c83b-1b5a-4d64-896a-8d1f68bef575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:11.812146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0347c83b-1b5a-4d64-896a-8d1f68bef575 None None] MFA Rules not processed for user `2335e75507f44afb8d77f9b70a668a94`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:11.813359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0347c83b-1b5a-4d64-896a-8d1f68bef575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:11.895756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:11.919164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 698/1399] 10.222.0.42 () {58 vars in 1040 bytes} [Wed Jul 20 17:42:11 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:11.929478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba1fab-3fa8-42db-a746-33d64b84bb5e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:11.930931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-00ba1fab-3fa8-42db-a746-33d64b84bb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:12.076559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba1fab-3fa8-42db-a746-33d64b84bb5e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:12.113066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:12.181652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:12.182248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:12.182532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:12.182808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:12.184564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:12.265425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:12.267079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:12.270001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b73bce-1160-4f41-b3c7-e1fed31e2597 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:12.270290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b73bce-1160-4f41-b3c7-e1fed31e2597 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:12.270779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b73bce-1160-4f41-b3c7-e1fed31e2597 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:12.272788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 699/1400] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:42:12.276040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 702/1401] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:42:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:42:12.278039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebf6b4-bd7f-44a0-a873-10c6fef65317 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:12.278252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebf6b4-bd7f-44a0-a873-10c6fef65317 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:12.278534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebf6b4-bd7f-44a0-a873-10c6fef65317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:12.354063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-79ebf6b4-bd7f-44a0-a873-10c6fef65317 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:12.355364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-79ebf6b4-bd7f-44a0-a873-10c6fef65317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:12.452300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 700/1402] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:12.497253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:12.498469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:12.562473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:12.589155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7efa1-ab4b-419f-bbfc-1cdf83d78465 service swift] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:12.702763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53741ad-18d3-4aa3-98c7-4f17a950bd25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:12.703353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53741ad-18d3-4aa3-98c7-4f17a950bd25 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:12.703636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53741ad-18d3-4aa3-98c7-4f17a950bd25 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:12.703913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53741ad-18d3-4aa3-98c7-4f17a950bd25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:12.705569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b53741ad-18d3-4aa3-98c7-4f17a950bd25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:12.806211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:12.810276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53741ad-18d3-4aa3-98c7-4f17a950bd25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:12.811897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53741ad-18d3-4aa3-98c7-4f17a950bd25 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:12.830550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 703/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:15.080779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253676-be58-49fc-b1a4-ea3aef71933a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:15.081382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253676-be58-49fc-b1a4-ea3aef71933a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:15.081382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253676-be58-49fc-b1a4-ea3aef71933a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:15.082715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 701/1404] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:15.088020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb205f81-90d6-4e7a-af7f-5db4ba20b7cb None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:15.088495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb205f81-90d6-4e7a-af7f-5db4ba20b7cb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:15.088652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb205f81-90d6-4e7a-af7f-5db4ba20b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:15.158412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-eb205f81-90d6-4e7a-af7f-5db4ba20b7cb None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:15.159635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-eb205f81-90d6-4e7a-af7f-5db4ba20b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:15.266327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 704/1405] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:15.312030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3253676-be58-49fc-b1a4-ea3aef71933a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:15.316086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c3253676-be58-49fc-b1a4-ea3aef71933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:15.390320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:15.477480 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3253676-be58-49fc-b1a4-ea3aef71933a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:15.601503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27d25f-b2ad-4202-8493-c8b30f8333cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:15.602190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d25f-b2ad-4202-8493-c8b30f8333cd service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:15.602467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d25f-b2ad-4202-8493-c8b30f8333cd service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:15.602934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27d25f-b2ad-4202-8493-c8b30f8333cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:15.604893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2a27d25f-b2ad-4202-8493-c8b30f8333cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:15.630736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:15.710025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27d25f-b2ad-4202-8493-c8b30f8333cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:15.711647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27d25f-b2ad-4202-8493-c8b30f8333cd service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:15.732531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 702/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:17.975713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b630645-f984-4ea9-bf10-c87bf38082a4 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:17.976294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b630645-f984-4ea9-bf10-c87bf38082a4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:17.976294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b630645-f984-4ea9-bf10-c87bf38082a4 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:17.977683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 705/1407] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:42:17.982924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb407e-53b0-4842-8403-646cc2aedaf9 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:17.983391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb407e-53b0-4842-8403-646cc2aedaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:17.983483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb407e-53b0-4842-8403-646cc2aedaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:18.053507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3bbb407e-53b0-4842-8403-646cc2aedaf9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:18.054745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3bbb407e-53b0-4842-8403-646cc2aedaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:18.165189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:18.184465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 703/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:18.221699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b630645-f984-4ea9-bf10-c87bf38082a4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:18.224621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8b630645-f984-4ea9-bf10-c87bf38082a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:18.297928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:18.391236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b630645-f984-4ea9-bf10-c87bf38082a4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:18.558400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a30cfe-9c0b-4b8a-a280-427d4225a799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:18.559018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a30cfe-9c0b-4b8a-a280-427d4225a799 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:18.559313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a30cfe-9c0b-4b8a-a280-427d4225a799 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:18.559575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a30cfe-9c0b-4b8a-a280-427d4225a799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:18.561332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-61a30cfe-9c0b-4b8a-a280-427d4225a799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:18.605251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:18.703192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a30cfe-9c0b-4b8a-a280-427d4225a799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:18.704866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a30cfe-9c0b-4b8a-a280-427d4225a799 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:18.726610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 706/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:28.724028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=34 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:36.163277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac30f5-767c-4a8d-80b0-69cd2bb96d3e None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:36.163850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac30f5-767c-4a8d-80b0-69cd2bb96d3e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:36.163850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac30f5-767c-4a8d-80b0-69cd2bb96d3e None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:36.165321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 704/1410] 10.222.0.42 () {54 vars in 918 bytes} [Wed Jul 20 17:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:36.171241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c16b-da9e-43a6-925f-4c3d4e984a32 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:36.171661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c16b-da9e-43a6-925f-4c3d4e984a32 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:36.171801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c16b-da9e-43a6-925f-4c3d4e984a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:36.244641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6917c16b-da9e-43a6-925f-4c3d4e984a32 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:36.245836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6917c16b-da9e-43a6-925f-4c3d4e984a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:36.348718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 707/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 20 17:42:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:36.363396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176c5ba-6003-4aaa-a59d-50e1668de020 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:36.363682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176c5ba-6003-4aaa-a59d-50e1668de020 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:36.363959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176c5ba-6003-4aaa-a59d-50e1668de020 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:36.365393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 705/1412] 10.222.0.42 () {54 vars in 918 bytes} [Wed Jul 20 17:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:36.369998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220650-1f7a-4b7c-9f8e-2ddc38d80fea None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:36.370291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220650-1f7a-4b7c-9f8e-2ddc38d80fea None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:36.370574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220650-1f7a-4b7c-9f8e-2ddc38d80fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:36.440292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:36.443473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-be220650-1f7a-4b7c-9f8e-2ddc38d80fea None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:36.444718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-be220650-1f7a-4b7c-9f8e-2ddc38d80fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:36.556415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 708/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 20 17:42:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:39.148509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd00b67-9c4b-4842-9d8d-79b5bf067e33 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:39.149035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd00b67-9c4b-4842-9d8d-79b5bf067e33 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:39.149149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd00b67-9c4b-4842-9d8d-79b5bf067e33 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:39.150974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:39.156181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be580-9f1e-431f-9621-e6d26b19dbc5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:39.156593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be580-9f1e-431f-9621-e6d26b19dbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:39.156788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7be580-9f1e-431f-9621-e6d26b19dbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:39.220573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6d7be580-9f1e-431f-9621-e6d26b19dbc5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:39.221788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6d7be580-9f1e-431f-9621-e6d26b19dbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:39.274760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:39.316060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:39.359531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd00b67-9c4b-4842-9d8d-79b5bf067e33 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:39.361051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9fd00b67-9c4b-4842-9d8d-79b5bf067e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:39.508151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd00b67-9c4b-4842-9d8d-79b5bf067e33 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:39.589019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:39.627021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa917ce1-4fde-45a6-90ff-34299d40a5e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:39.627641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa917ce1-4fde-45a6-90ff-34299d40a5e8 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:39.627927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa917ce1-4fde-45a6-90ff-34299d40a5e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:39.628205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa917ce1-4fde-45a6-90ff-34299d40a5e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:39.630626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-aa917ce1-4fde-45a6-90ff-34299d40a5e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:39.724882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa917ce1-4fde-45a6-90ff-34299d40a5e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:39.726536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa917ce1-4fde-45a6-90ff-34299d40a5e8 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:39.744088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 707/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:40.637760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fa7ac-edac-4c7d-8231-a71084442d3b None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:40.638334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fa7ac-edac-4c7d-8231-a71084442d3b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:40.638334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fa7ac-edac-4c7d-8231-a71084442d3b None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:40.639740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:40.644617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74d22c-84f8-47ba-84c0-ef4687033060 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:40.645045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74d22c-84f8-47ba-84c0-ef4687033060 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:40.645180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74d22c-84f8-47ba-84c0-ef4687033060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:40.719552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ee74d22c-84f8-47ba-84c0-ef4687033060 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:40.720797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ee74d22c-84f8-47ba-84c0-ef4687033060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:40.747401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:40.817702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:40.968224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fa7ac-edac-4c7d-8231-a71084442d3b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:40.969790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e28fa7ac-edac-4c7d-8231-a71084442d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:41.111849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fa7ac-edac-4c7d-8231-a71084442d3b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:41.151847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:41.224097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa50ab-758d-4e5a-ba44-b5c0c53ed16e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:41.224690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa50ab-758d-4e5a-ba44-b5c0c53ed16e service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:41.224968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa50ab-758d-4e5a-ba44-b5c0c53ed16e service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:41.225244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa50ab-758d-4e5a-ba44-b5c0c53ed16e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:41.226893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9caa50ab-758d-4e5a-ba44-b5c0c53ed16e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:41.313606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa50ab-758d-4e5a-ba44-b5c0c53ed16e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:41.315210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa50ab-758d-4e5a-ba44-b5c0c53ed16e service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:41.333606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 711/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:42.227387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a38980-d68d-4a09-a994-bac41a426010 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:42.227942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a38980-d68d-4a09-a994-bac41a426010 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:42.227942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a38980-d68d-4a09-a994-bac41a426010 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:42.229917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:42.234913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175737-bfbe-4286-b175-1765c15eda83 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:42.235310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175737-bfbe-4286-b175-1765c15eda83 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:42.235497 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175737-bfbe-4286-b175-1765c15eda83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:42.286229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:42.307616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b7175737-bfbe-4286-b175-1765c15eda83 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:42.308817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b7175737-bfbe-4286-b175-1765c15eda83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:42.403797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:42.553526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a38980-d68d-4a09-a994-bac41a426010 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:42.555071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-93a38980-d68d-4a09-a994-bac41a426010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:42.698387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a38980-d68d-4a09-a994-bac41a426010 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:42.711452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:42.811366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b26d2f-469d-41b5-b464-af077a2c4e90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:42.812016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b26d2f-469d-41b5-b464-af077a2c4e90 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:42.812299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b26d2f-469d-41b5-b464-af077a2c4e90 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:42.812605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b26d2f-469d-41b5-b464-af077a2c4e90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:42.814253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-48b26d2f-469d-41b5-b464-af077a2c4e90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:42.901430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b26d2f-469d-41b5-b464-af077a2c4e90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:42.903024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b26d2f-469d-41b5-b464-af077a2c4e90 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:42.920911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 710/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:43.773731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72be53-9029-4cfe-b6cd-ebb04dcdece3 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:43.774217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72be53-9029-4cfe-b6cd-ebb04dcdece3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:43.774331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72be53-9029-4cfe-b6cd-ebb04dcdece3 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:43.775707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:43.780665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50113919-6a04-44e8-aaec-127fc37f7880 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:43.781111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50113919-6a04-44e8-aaec-127fc37f7880 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:43.781220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50113919-6a04-44e8-aaec-127fc37f7880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:43.799300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:43.853863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-50113919-6a04-44e8-aaec-127fc37f7880 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:43.855058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-50113919-6a04-44e8-aaec-127fc37f7880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:43.951419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:44.100062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72be53-9029-4cfe-b6cd-ebb04dcdece3 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:44.101736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bc72be53-9029-4cfe-b6cd-ebb04dcdece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:44.210210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:44.246808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72be53-9029-4cfe-b6cd-ebb04dcdece3 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:44.359740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d940ae-91a0-45a0-9142-abe8bfd3b2dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:44.360401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d940ae-91a0-45a0-9142-abe8bfd3b2dc service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:44.360715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d940ae-91a0-45a0-9142-abe8bfd3b2dc service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:44.360988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d940ae-91a0-45a0-9142-abe8bfd3b2dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:44.362644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-82d940ae-91a0-45a0-9142-abe8bfd3b2dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:44.437964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:44.451372 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d940ae-91a0-45a0-9142-abe8bfd3b2dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:44.453013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d940ae-91a0-45a0-9142-abe8bfd3b2dc service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:44.471142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 714/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:45.320690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcca17-a3b3-4946-8b6b-ac618279b416 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:45.321247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcca17-a3b3-4946-8b6b-ac618279b416 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:45.321356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcca17-a3b3-4946-8b6b-ac618279b416 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:45.323283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:45.328085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a84a7-aed1-4f43-83a0-bc593e04aee6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:45.328550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a84a7-aed1-4f43-83a0-bc593e04aee6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:45.328684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a84a7-aed1-4f43-83a0-bc593e04aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:45.401972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ed6a84a7-aed1-4f43-83a0-bc593e04aee6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:45.403098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ed6a84a7-aed1-4f43-83a0-bc593e04aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:45.497300 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:45.653898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcca17-a3b3-4946-8b6b-ac618279b416 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:45.655341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-05bcca17-a3b3-4946-8b6b-ac618279b416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:45.737416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:45.798078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcca17-a3b3-4946-8b6b-ac618279b416 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:45.907532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf174b0-d812-4e02-8d6c-f9f9110b8069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:45.908110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf174b0-d812-4e02-8d6c-f9f9110b8069 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:45.908393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf174b0-d812-4e02-8d6c-f9f9110b8069 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:45.908702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf174b0-d812-4e02-8d6c-f9f9110b8069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:45.910339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1cf174b0-d812-4e02-8d6c-f9f9110b8069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:45.957851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:46.000686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf174b0-d812-4e02-8d6c-f9f9110b8069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:46.002387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf174b0-d812-4e02-8d6c-f9f9110b8069 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:46.020600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 713/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:46.898477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21332c7-a0d8-415a-86fa-20c21ecc2261 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:46.898989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21332c7-a0d8-415a-86fa-20c21ecc2261 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:46.899085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21332c7-a0d8-415a-86fa-20c21ecc2261 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:46.900451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:46.905467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b96859-3697-454d-b965-fada59ebdca0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:46.905894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b96859-3697-454d-b965-fada59ebdca0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:46.906036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b96859-3697-454d-b965-fada59ebdca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:46.980488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-28b96859-3697-454d-b965-fada59ebdca0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:46.981706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-28b96859-3697-454d-b965-fada59ebdca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:47.081382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:47.231154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21332c7-a0d8-415a-86fa-20c21ecc2261 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:47.232774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b21332c7-a0d8-415a-86fa-20c21ecc2261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:47.281470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:47.378211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21332c7-a0d8-415a-86fa-20c21ecc2261 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:47.487534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608553a-93f2-45ef-8197-69132a21cb12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:47.488174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608553a-93f2-45ef-8197-69132a21cb12 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:47.488478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608553a-93f2-45ef-8197-69132a21cb12 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:47.488758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608553a-93f2-45ef-8197-69132a21cb12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:47.490389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e608553a-93f2-45ef-8197-69132a21cb12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:47.497082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:47.577038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608553a-93f2-45ef-8197-69132a21cb12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:47.578642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608553a-93f2-45ef-8197-69132a21cb12 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:47.596536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 717/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:48.527162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c623-9ead-419e-a456-3e1f93b01890 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:48.527727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c623-9ead-419e-a456-3e1f93b01890 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:48.527727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c623-9ead-419e-a456-3e1f93b01890 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:48.529118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:48.533921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf547d-e232-420b-a015-ba79599613d1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:48.534294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf547d-e232-420b-a015-ba79599613d1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:48.534487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf547d-e232-420b-a015-ba79599613d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:48.607235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-eacf547d-e232-420b-a015-ba79599613d1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:48.608463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-eacf547d-e232-420b-a015-ba79599613d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:48.673873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:48.702548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:48.851784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c623-9ead-419e-a456-3e1f93b01890 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:48.853233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0696c623-9ead-419e-a456-3e1f93b01890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:48.870866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:48.996055 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c623-9ead-419e-a456-3e1f93b01890 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:49.082286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:49.107705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329d0e7-98c8-4eba-a9cf-649a87376acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:49.108302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329d0e7-98c8-4eba-a9cf-649a87376acd service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:49.108625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329d0e7-98c8-4eba-a9cf-649a87376acd service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:49.108908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329d0e7-98c8-4eba-a9cf-649a87376acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:49.110641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f329d0e7-98c8-4eba-a9cf-649a87376acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:49.199636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329d0e7-98c8-4eba-a9cf-649a87376acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:49.201279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329d0e7-98c8-4eba-a9cf-649a87376acd service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:49.218845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 716/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:50.092635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b697c-73fa-4c98-ad56-8294c6a1de67 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:50.093214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b697c-73fa-4c98-ad56-8294c6a1de67 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:50.093851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b697c-73fa-4c98-ad56-8294c6a1de67 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:50.095494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:50.100663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616f8da-2b61-4404-8100-1f3ee7196789 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:50.101085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616f8da-2b61-4404-8100-1f3ee7196789 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:50.101231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616f8da-2b61-4404-8100-1f3ee7196789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:50.177775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8616f8da-2b61-4404-8100-1f3ee7196789 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:50.178998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8616f8da-2b61-4404-8100-1f3ee7196789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:50.220793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:50.275969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:50.426379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b697c-73fa-4c98-ad56-8294c6a1de67 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:50.428167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f87b697c-73fa-4c98-ad56-8294c6a1de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:50.575986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b697c-73fa-4c98-ad56-8294c6a1de67 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:50.631884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:50.691658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45ad74-8e7d-4801-946b-191f8ed52690 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:50.692224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ad74-8e7d-4801-946b-191f8ed52690 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:50.692561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ad74-8e7d-4801-946b-191f8ed52690 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:50.692831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45ad74-8e7d-4801-946b-191f8ed52690 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:50.694469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1c45ad74-8e7d-4801-946b-191f8ed52690 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:50.780785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45ad74-8e7d-4801-946b-191f8ed52690 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:50.782393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45ad74-8e7d-4801-946b-191f8ed52690 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:50.800771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 720/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:51.674554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204747-7a6b-49ad-b7bb-7c7d7036ff1f None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:51.675080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204747-7a6b-49ad-b7bb-7c7d7036ff1f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:51.675194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204747-7a6b-49ad-b7bb-7c7d7036ff1f None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:51.676794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:51.682743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14f777-7289-42d9-8a4b-42f97132be74 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:51.683180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14f777-7289-42d9-8a4b-42f97132be74 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:51.683311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14f777-7289-42d9-8a4b-42f97132be74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:51.760173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4c14f777-7289-42d9-8a4b-42f97132be74 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:51.761348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4c14f777-7289-42d9-8a4b-42f97132be74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:51.763994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:51.856340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:52.005448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204747-7a6b-49ad-b7bb-7c7d7036ff1f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:52.007073 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-54204747-7a6b-49ad-b7bb-7c7d7036ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:52.150476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204747-7a6b-49ad-b7bb-7c7d7036ff1f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:52.176972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:52.261555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcc723-ed7b-472b-94a4-36f729b04721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:52.262210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcc723-ed7b-472b-94a4-36f729b04721 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:52.262496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcc723-ed7b-472b-94a4-36f729b04721 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:52.262776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcc723-ed7b-472b-94a4-36f729b04721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:52.264421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-12fcc723-ed7b-472b-94a4-36f729b04721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:52.351499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcc723-ed7b-472b-94a4-36f729b04721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:52.353041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcc723-ed7b-472b-94a4-36f729b04721 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:52.371153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 719/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:42:54.303533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea55c6-11f2-4a48-b4ac-b963c1462c5a None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:54.304062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea55c6-11f2-4a48-b4ac-b963c1462c5a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:54.304155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea55c6-11f2-4a48-b4ac-b963c1462c5a None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:54.305557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:54.310742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c91360-9403-493d-a825-59d9bb908466 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:54.311150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c91360-9403-493d-a825-59d9bb908466 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:54.311305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c91360-9403-493d-a825-59d9bb908466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:54.355225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:54.379793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b3c91360-9403-493d-a825-59d9bb908466 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:54.381032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b3c91360-9403-493d-a825-59d9bb908466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:54.477709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:54.516326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea55c6-11f2-4a48-b4ac-b963c1462c5a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:54.517878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ebea55c6-11f2-4a48-b4ac-b963c1462c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:54.619843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea55c6-11f2-4a48-b4ac-b963c1462c5a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:54.651434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:54.692913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c34f-4129-4f11-ae92-22d39336c153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:54.693737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c34f-4129-4f11-ae92-22d39336c153 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:54.694022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c34f-4129-4f11-ae92-22d39336c153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:54.694517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c34f-4129-4f11-ae92-22d39336c153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:54.696525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3bc8c34f-4129-4f11-ae92-22d39336c153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:54.802389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8c34f-4129-4f11-ae92-22d39336c153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:54.803828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8c34f-4129-4f11-ae92-22d39336c153 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:54.822463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 723/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:42:57.089787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d67542-e712-4168-90a3-ab386fa3c3d4 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:57.090925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d67542-e712-4168-90a3-ab386fa3c3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:57.090925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d67542-e712-4168-90a3-ab386fa3c3d4 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:57.092259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:42:57.097605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2c509-35e0-4b32-997b-693561c3bd07 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:57.098049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2c509-35e0-4b32-997b-693561c3bd07 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:57.098158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2c509-35e0-4b32-997b-693561c3bd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:57.162572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-17f2c509-35e0-4b32-997b-693561c3bd07 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:42:57.163773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-17f2c509-35e0-4b32-997b-693561c3bd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:57.177118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:57.258713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:42:57.297242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d67542-e712-4168-90a3-ab386fa3c3d4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:42:57.304583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-63d67542-e712-4168-90a3-ab386fa3c3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:57.401365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d67542-e712-4168-90a3-ab386fa3c3d4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:42:57.413869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:42:57.476311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a878b2-5d51-4e6b-8f90-739d9e319875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:42:57.482021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a878b2-5d51-4e6b-8f90-739d9e319875 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:42:57.482021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a878b2-5d51-4e6b-8f90-739d9e319875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:42:57.482021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a878b2-5d51-4e6b-8f90-739d9e319875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:42:57.482021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-82a878b2-5d51-4e6b-8f90-739d9e319875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:42:57.584715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a878b2-5d51-4e6b-8f90-739d9e319875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:42:57.586139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a878b2-5d51-4e6b-8f90-739d9e319875 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:42:57.604154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 722/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:00.041116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23f026-fae9-49ff-8291-95e592d7f65d None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:00.041654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23f026-fae9-49ff-8291-95e592d7f65d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:00.041654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23f026-fae9-49ff-8291-95e592d7f65d None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:00.042976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:00.048972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be2ad-806e-45db-8e06-fe09d6b60e3e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:00.049393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be2ad-806e-45db-8e06-fe09d6b60e3e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:00.049529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be2ad-806e-45db-8e06-fe09d6b60e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:00.098906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:00.116982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-9d7be2ad-806e-45db-8e06-fe09d6b60e3e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:00.118192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9d7be2ad-806e-45db-8e06-fe09d6b60e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:00.212515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:00.251811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23f026-fae9-49ff-8291-95e592d7f65d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:00.253346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1b23f026-fae9-49ff-8291-95e592d7f65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:00.354124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23f026-fae9-49ff-8291-95e592d7f65d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:00.393009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:00.425635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704df55-2012-45c4-87fe-3f490e09d28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:00.426279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704df55-2012-45c4-87fe-3f490e09d28d service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:00.426566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704df55-2012-45c4-87fe-3f490e09d28d service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:00.427098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704df55-2012-45c4-87fe-3f490e09d28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:00.428766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2704df55-2012-45c4-87fe-3f490e09d28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:00.535604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704df55-2012-45c4-87fe-3f490e09d28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:00.537095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704df55-2012-45c4-87fe-3f490e09d28d service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:00.554729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 726/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:02.686489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2197c-89cc-4b91-9a56-873f82938d0b None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:02.687049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2197c-89cc-4b91-9a56-873f82938d0b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:02.687049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2197c-89cc-4b91-9a56-873f82938d0b None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:02.688422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:02.693950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20156cc-074a-465c-9099-c91ef80ab8a5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:02.694431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20156cc-074a-465c-9099-c91ef80ab8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:02.694536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20156cc-074a-465c-9099-c91ef80ab8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:02.758195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d20156cc-074a-465c-9099-c91ef80ab8a5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:02.759409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d20156cc-074a-465c-9099-c91ef80ab8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:02.780588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:02.859966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:02.898145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2197c-89cc-4b91-9a56-873f82938d0b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:02.899680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-dee2197c-89cc-4b91-9a56-873f82938d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:03.001535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2197c-89cc-4b91-9a56-873f82938d0b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:03.021914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:03.076007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442351-3781-4600-bf53-5b7f3b5a2546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:03.076698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442351-3781-4600-bf53-5b7f3b5a2546 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:03.076987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442351-3781-4600-bf53-5b7f3b5a2546 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:03.077263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442351-3781-4600-bf53-5b7f3b5a2546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:03.078932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9e442351-3781-4600-bf53-5b7f3b5a2546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:03.186428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442351-3781-4600-bf53-5b7f3b5a2546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:03.188008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442351-3781-4600-bf53-5b7f3b5a2546 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:03.206208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 725/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:05.272943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd494c37-7aa6-4f5e-b003-84a0e23bb50d None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:05.273521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd494c37-7aa6-4f5e-b003-84a0e23bb50d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:05.273521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd494c37-7aa6-4f5e-b003-84a0e23bb50d None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:05.274875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:05.280690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7d81d-b4d7-472a-a221-617a82929d13 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:05.281083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7d81d-b4d7-472a-a221-617a82929d13 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:05.281248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7d81d-b4d7-472a-a221-617a82929d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:05.342030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:05.344577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-60f7d81d-b4d7-472a-a221-617a82929d13 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:05.345803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-60f7d81d-b4d7-472a-a221-617a82929d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:05.439679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:05.477275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd494c37-7aa6-4f5e-b003-84a0e23bb50d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:05.478825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fd494c37-7aa6-4f5e-b003-84a0e23bb50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:05.582657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd494c37-7aa6-4f5e-b003-84a0e23bb50d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:05.630372 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:05.655315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5732d-48eb-4cb6-9f41-6f4d113a5e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:05.655954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5732d-48eb-4cb6-9f41-6f4d113a5e53 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:05.656237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5732d-48eb-4cb6-9f41-6f4d113a5e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:05.656537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5732d-48eb-4cb6-9f41-6f4d113a5e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:05.658208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0fb5732d-48eb-4cb6-9f41-6f4d113a5e53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:05.764420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5732d-48eb-4cb6-9f41-6f4d113a5e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:05.765886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5732d-48eb-4cb6-9f41-6f4d113a5e53 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:05.785472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 729/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:07.868059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3d2c-0797-48ac-a98c-cfe6fa2b86fb None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:07.868675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3d2c-0797-48ac-a98c-cfe6fa2b86fb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:07.868814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3d2c-0797-48ac-a98c-cfe6fa2b86fb None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:07.870294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:07.876278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf88fdd-fb86-45d3-8384-9f9a47a46a04 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:07.876749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf88fdd-fb86-45d3-8384-9f9a47a46a04 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:07.876878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf88fdd-fb86-45d3-8384-9f9a47a46a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:07.947300 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3bf88fdd-fb86-45d3-8384-9f9a47a46a04 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:07.948553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3bf88fdd-fb86-45d3-8384-9f9a47a46a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:07.976368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:08.042110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:08.081129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3d2c-0797-48ac-a98c-cfe6fa2b86fb None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:08.082704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-508b3d2c-0797-48ac-a98c-cfe6fa2b86fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:08.184014 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3d2c-0797-48ac-a98c-cfe6fa2b86fb None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:08.212988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:08.257764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897612b-025c-42c1-8499-e466d85bfc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:08.258345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897612b-025c-42c1-8499-e466d85bfc42 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:08.258630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897612b-025c-42c1-8499-e466d85bfc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:08.258909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897612b-025c-42c1-8499-e466d85bfc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:08.260587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8897612b-025c-42c1-8499-e466d85bfc42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:08.367243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897612b-025c-42c1-8499-e466d85bfc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:08.368760 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897612b-025c-42c1-8499-e466d85bfc42 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:08.386777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 728/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:10.481203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858ce45-4c37-4f1b-aad0-1741c77f8f07 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:10.481757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858ce45-4c37-4f1b-aad0-1741c77f8f07 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:10.481757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858ce45-4c37-4f1b-aad0-1741c77f8f07 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:10.483159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:43:10.488300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6c87d-d298-469e-bebf-f70a0ad11d3e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:10.488749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6c87d-d298-469e-bebf-f70a0ad11d3e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:10.488905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6c87d-d298-469e-bebf-f70a0ad11d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:10.557254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-6cd6c87d-d298-469e-bebf-f70a0ad11d3e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:10.558472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6cd6c87d-d298-469e-bebf-f70a0ad11d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:10.567300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:10.655798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:10.693406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858ce45-4c37-4f1b-aad0-1741c77f8f07 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:10.694936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5858ce45-4c37-4f1b-aad0-1741c77f8f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:10.795068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858ce45-4c37-4f1b-aad0-1741c77f8f07 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:10.850075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:10.868513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8d662-ca42-4c7a-a4a8-f8feb7dea320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:10.869160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8d662-ca42-4c7a-a4a8-f8feb7dea320 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:10.869483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8d662-ca42-4c7a-a4a8-f8feb7dea320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:10.869764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8d662-ca42-4c7a-a4a8-f8feb7dea320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:10.871397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a3e8d662-ca42-4c7a-a4a8-f8feb7dea320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:10.977376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8d662-ca42-4c7a-a4a8-f8feb7dea320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:10.978856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8d662-ca42-4c7a-a4a8-f8feb7dea320 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:10.998082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 732/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:13.076174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3125d-dd41-42b6-8132-a1e2caf6aafc None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:13.076765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3125d-dd41-42b6-8132-a1e2caf6aafc None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:13.076765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3125d-dd41-42b6-8132-a1e2caf6aafc None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:13.078140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:13.083426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e189db2-30be-44cf-a267-d1d43b2f7a19 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:13.083867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e189db2-30be-44cf-a267-d1d43b2f7a19 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:13.083992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e189db2-30be-44cf-a267-d1d43b2f7a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:13.149667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0e189db2-30be-44cf-a267-d1d43b2f7a19 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:13.150884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0e189db2-30be-44cf-a267-d1d43b2f7a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:13.189389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:13.252424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:13.290380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3125d-dd41-42b6-8132-a1e2caf6aafc None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:13.291933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8bc3125d-dd41-42b6-8132-a1e2caf6aafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:13.393378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3125d-dd41-42b6-8132-a1e2caf6aafc None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:13.430320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:13.465930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec9c0-b2ca-4121-a4c4-020893da508c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:13.466577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec9c0-b2ca-4121-a4c4-020893da508c service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:13.466929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec9c0-b2ca-4121-a4c4-020893da508c service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:13.467733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec9c0-b2ca-4121-a4c4-020893da508c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:13.469413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-50dec9c0-b2ca-4121-a4c4-020893da508c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:13.583804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec9c0-b2ca-4121-a4c4-020893da508c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:13.585286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec9c0-b2ca-4121-a4c4-020893da508c service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:13.603973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 731/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:15.626705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cb09a-9be7-45e8-ad32-0c491235b2a9 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:15.627276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cb09a-9be7-45e8-ad32-0c491235b2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:15.627276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cb09a-9be7-45e8-ad32-0c491235b2a9 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:15.628662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:15.633812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ff642-0f4c-4ffe-993b-7deb56f124a8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:15.634248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ff642-0f4c-4ffe-993b-7deb56f124a8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:15.634352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ff642-0f4c-4ffe-993b-7deb56f124a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:15.696803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cd8ff642-0f4c-4ffe-993b-7deb56f124a8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:15.698006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cd8ff642-0f4c-4ffe-993b-7deb56f124a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:15.715862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:15.791890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:15.829125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cb09a-9be7-45e8-ad32-0c491235b2a9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:15.830672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-394cb09a-9be7-45e8-ad32-0c491235b2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:15.934567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cb09a-9be7-45e8-ad32-0c491235b2a9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:16.001760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:16.008292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084d0c9-5274-43b9-b7bb-f5d4009dca9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:16.008969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d0c9-5274-43b9-b7bb-f5d4009dca9f service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:16.009253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d0c9-5274-43b9-b7bb-f5d4009dca9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:16.009530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d0c9-5274-43b9-b7bb-f5d4009dca9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:16.011236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9084d0c9-5274-43b9-b7bb-f5d4009dca9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:16.119022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084d0c9-5274-43b9-b7bb-f5d4009dca9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:16.120450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084d0c9-5274-43b9-b7bb-f5d4009dca9f service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:16.138147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 735/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:18.167045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb1a77-408e-4644-b00f-28ac01d74592 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:18.167571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb1a77-408e-4644-b00f-28ac01d74592 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:18.167689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb1a77-408e-4644-b00f-28ac01d74592 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:18.169124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:18.174935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018db8fc-2d10-4adc-a91c-6bd6ec99d017 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:18.175400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018db8fc-2d10-4adc-a91c-6bd6ec99d017 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:18.175536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018db8fc-2d10-4adc-a91c-6bd6ec99d017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:18.239653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-018db8fc-2d10-4adc-a91c-6bd6ec99d017 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:18.240905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-018db8fc-2d10-4adc-a91c-6bd6ec99d017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:18.289143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:18.337159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:18.374020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb1a77-408e-4644-b00f-28ac01d74592 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:18.375566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e0eb1a77-408e-4644-b00f-28ac01d74592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:18.475214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb1a77-408e-4644-b00f-28ac01d74592 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:18.520561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:18.546467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc9c44-6ffc-44ec-9160-d7fa0fb281b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:18.547167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc9c44-6ffc-44ec-9160-d7fa0fb281b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:18.547447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc9c44-6ffc-44ec-9160-d7fa0fb281b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:18.547873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc9c44-6ffc-44ec-9160-d7fa0fb281b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:18.549530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6fcc9c44-6ffc-44ec-9160-d7fa0fb281b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:18.653713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc9c44-6ffc-44ec-9160-d7fa0fb281b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:18.655633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc9c44-6ffc-44ec-9160-d7fa0fb281b1 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:18.675206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 734/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:43:28.335168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=22 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:28.672959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=66 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:38.337626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547e93a-e24b-4b3d-9554-2af193eff242 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:38.338200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547e93a-e24b-4b3d-9554-2af193eff242 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:38.338200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547e93a-e24b-4b3d-9554-2af193eff242 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:38.339589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 737/1471] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:38.344779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118fe0e-55c9-4ba2-bb90-a7e5bfbf9a9d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:38.345210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118fe0e-55c9-4ba2-bb90-a7e5bfbf9a9d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:38.345319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118fe0e-55c9-4ba2-bb90-a7e5bfbf9a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:38.407919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-2118fe0e-55c9-4ba2-bb90-a7e5bfbf9a9d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:38.409045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2118fe0e-55c9-4ba2-bb90-a7e5bfbf9a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:38.502720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 735/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:38.584286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f65ef-7a3a-4650-b583-d6194422d07e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:38.584612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f65ef-7a3a-4650-b583-d6194422d07e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:38.584896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f65ef-7a3a-4650-b583-d6194422d07e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:38.586310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 738/1473] 10.222.0.42 () {54 vars in 966 bytes} [Wed Jul 20 17:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:38.591131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff11e5-dc4a-4962-b864-24aa856853a8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:38.591423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff11e5-dc4a-4962-b864-24aa856853a8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:38.592193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff11e5-dc4a-4962-b864-24aa856853a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:38.664449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-feff11e5-dc4a-4962-b864-24aa856853a8 None None] MFA Rules not processed for user `de1a294634524a9d8ff48c7c50726228`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:38.665689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-feff11e5-dc4a-4962-b864-24aa856853a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:38.686217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:38.741258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 736/1474] 10.222.0.42 () {58 vars in 1063 bytes} [Wed Jul 20 17:43:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:38.748697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f65ef-7a3a-4650-b583-d6194422d07e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:38.750217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-159f65ef-7a3a-4650-b583-d6194422d07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:38.850999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f65ef-7a3a-4650-b583-d6194422d07e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:38.922421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc6626-5a5e-44ab-aad3-bdec91b2b7bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:38.923017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc6626-5a5e-44ab-aad3-bdec91b2b7bb service glance] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:38.923299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc6626-5a5e-44ab-aad3-bdec91b2b7bb service glance] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:38.923573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc6626-5a5e-44ab-aad3-bdec91b2b7bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:38.925308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5dbc6626-5a5e-44ab-aad3-bdec91b2b7bb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:38.978259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:39.035268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc6626-5a5e-44ab-aad3-bdec91b2b7bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:39.036741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc6626-5a5e-44ab-aad3-bdec91b2b7bb service glance] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:39.054965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 739/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:41.180424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452cb44-f922-48c1-85dc-b0b84b54d12b None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:41.181035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452cb44-f922-48c1-85dc-b0b84b54d12b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:41.181035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452cb44-f922-48c1-85dc-b0b84b54d12b None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:41.182415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 737/1476] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:41.188147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956bfe5-62ce-4c40-9bb4-f694159db5ff None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:41.188563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956bfe5-62ce-4c40-9bb4-f694159db5ff None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:41.188755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956bfe5-62ce-4c40-9bb4-f694159db5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:41.251738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1956bfe5-62ce-4c40-9bb4-f694159db5ff None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:41.252950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1956bfe5-62ce-4c40-9bb4-f694159db5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:41.348284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 740/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:41.371310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452cb44-f922-48c1-85dc-b0b84b54d12b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:41.377542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4452cb44-f922-48c1-85dc-b0b84b54d12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:41.495543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452cb44-f922-48c1-85dc-b0b84b54d12b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:41.545672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:41.588546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680d3bf-b352-4bd0-928a-a6b5cf55faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:41.589366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680d3bf-b352-4bd0-928a-a6b5cf55faa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:41.589629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680d3bf-b352-4bd0-928a-a6b5cf55faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:41.589902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680d3bf-b352-4bd0-928a-a6b5cf55faa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:41.590628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680d3bf-b352-4bd0-928a-a6b5cf55faa9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:41.592967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-6680d3bf-b352-4bd0-928a-a6b5cf55faa9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:43:41.592967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:43:41.592967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:43:41.594106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680d3bf-b352-4bd0-928a-a6b5cf55faa9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:41.612942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 738/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Jul 20 17:43:41 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:43:43.648963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da121c32-8b9d-4d31-b190-ef2cda2dc79b None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:43.649526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da121c32-8b9d-4d31-b190-ef2cda2dc79b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:43.649526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da121c32-8b9d-4d31-b190-ef2cda2dc79b None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:43.650931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 741/1479] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:43.656904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e0002-23f4-4bde-a071-edee21b40bf8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:43.657302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e0002-23f4-4bde-a071-edee21b40bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:43.657469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e0002-23f4-4bde-a071-edee21b40bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:43.725685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-795e0002-23f4-4bde-a071-edee21b40bf8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:43.726908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-795e0002-23f4-4bde-a071-edee21b40bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:43.821444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 739/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:43.864815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da121c32-8b9d-4d31-b190-ef2cda2dc79b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:43.866486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-da121c32-8b9d-4d31-b190-ef2cda2dc79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:43.876889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:44.011156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da121c32-8b9d-4d31-b190-ef2cda2dc79b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:44.091787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:44.126664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b108ee8-2089-4bd8-9566-d4ebbbeafb9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:44.127311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108ee8-2089-4bd8-9566-d4ebbbeafb9a service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:44.127600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108ee8-2089-4bd8-9566-d4ebbbeafb9a service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:44.127878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b108ee8-2089-4bd8-9566-d4ebbbeafb9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:44.129572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0b108ee8-2089-4bd8-9566-d4ebbbeafb9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:44.223331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b108ee8-2089-4bd8-9566-d4ebbbeafb9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:44.224963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b108ee8-2089-4bd8-9566-d4ebbbeafb9a service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:44.243179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 742/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:46.458882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817306a-e556-42b8-97f5-0ebe8aaaa00c None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:46.459394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817306a-e556-42b8-97f5-0ebe8aaaa00c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:46.459507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817306a-e556-42b8-97f5-0ebe8aaaa00c None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:46.460896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 740/1482] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:46.466070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa5ac4-4862-4d29-9b85-cb2964c39792 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:46.466510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa5ac4-4862-4d29-9b85-cb2964c39792 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:46.466841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa5ac4-4862-4d29-9b85-cb2964c39792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:46.530750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-91aa5ac4-4862-4d29-9b85-cb2964c39792 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:46.531938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-91aa5ac4-4862-4d29-9b85-cb2964c39792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:46.577534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:46.626738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 743/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:46.662492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817306a-e556-42b8-97f5-0ebe8aaaa00c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:46.664371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8817306a-e556-42b8-97f5-0ebe8aaaa00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:46.689085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:46.769207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817306a-e556-42b8-97f5-0ebe8aaaa00c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:46.842330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112f826-ea08-4eb0-8fc7-ef9a12981c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:46.842984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f826-ea08-4eb0-8fc7-ef9a12981c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:46.843273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f826-ea08-4eb0-8fc7-ef9a12981c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:46.843554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f826-ea08-4eb0-8fc7-ef9a12981c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:46.845270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4112f826-ea08-4eb0-8fc7-ef9a12981c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:46.919328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:46.953524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112f826-ea08-4eb0-8fc7-ef9a12981c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:46.954975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112f826-ea08-4eb0-8fc7-ef9a12981c38 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:46.974107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 741/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:49.066690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e8fe1-9d16-4164-a1f0-72cc57d5bbe8 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:49.067370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e8fe1-9d16-4164-a1f0-72cc57d5bbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:49.067370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e8fe1-9d16-4164-a1f0-72cc57d5bbe8 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:49.068697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 744/1485] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:49.073874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee1622-3b31-4a68-b357-ef3a68f2efef None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:49.074289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee1622-3b31-4a68-b357-ef3a68f2efef None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:49.074445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee1622-3b31-4a68-b357-ef3a68f2efef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:49.140536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-6eee1622-3b31-4a68-b357-ef3a68f2efef None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:49.141763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6eee1622-3b31-4a68-b357-ef3a68f2efef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:49.235636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 742/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:49.274641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e8fe1-9d16-4164-a1f0-72cc57d5bbe8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:49.276169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3f5e8fe1-9d16-4164-a1f0-72cc57d5bbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:49.496041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e8fe1-9d16-4164-a1f0-72cc57d5bbe8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:49.565437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:49.567138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e5257-9f7e-4ed7-bb04-7b51bc2bc294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:49.567767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e5257-9f7e-4ed7-bb04-7b51bc2bc294 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:49.568046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e5257-9f7e-4ed7-bb04-7b51bc2bc294 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:49.568319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e5257-9f7e-4ed7-bb04-7b51bc2bc294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:49.569969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3f5e5257-9f7e-4ed7-bb04-7b51bc2bc294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:49.676151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e5257-9f7e-4ed7-bb04-7b51bc2bc294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:49.677874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e5257-9f7e-4ed7-bb04-7b51bc2bc294 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:49.697957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 745/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:51.937212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613f913-a403-4e33-9e72-4a3b1fef52ec None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:51.937763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613f913-a403-4e33-9e72-4a3b1fef52ec None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:51.937763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613f913-a403-4e33-9e72-4a3b1fef52ec None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:51.939166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 743/1488] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:43:51.944583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c601-9bea-4c64-ba3d-331351ff5088 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:51.951238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c601-9bea-4c64-ba3d-331351ff5088 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:51.951238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c601-9bea-4c64-ba3d-331351ff5088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:52.007740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3d65c601-9bea-4c64-ba3d-331351ff5088 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:52.008907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3d65c601-9bea-4c64-ba3d-331351ff5088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:52.060969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:52.104835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 746/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:52.145831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613f913-a403-4e33-9e72-4a3b1fef52ec None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:52.147353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b613f913-a403-4e33-9e72-4a3b1fef52ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:52.184916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:52.252459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613f913-a403-4e33-9e72-4a3b1fef52ec None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:52.326821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc265b57-90ef-470d-b237-aca170f63b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:52.327393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc265b57-90ef-470d-b237-aca170f63b67 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:52.327681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc265b57-90ef-470d-b237-aca170f63b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:52.327960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc265b57-90ef-470d-b237-aca170f63b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:52.329644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-dc265b57-90ef-470d-b237-aca170f63b67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:52.407910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:52.436884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc265b57-90ef-470d-b237-aca170f63b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:52.438330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc265b57-90ef-470d-b237-aca170f63b67 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:52.458150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 744/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:43:54.996304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702e47c-9be7-48cf-b656-afbc6b0c62ad None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:54.996877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702e47c-9be7-48cf-b656-afbc6b0c62ad None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:54.996877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702e47c-9be7-48cf-b656-afbc6b0c62ad None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:54.998325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 747/1491] 10.222.0.42 () {54 vars in 926 bytes} [Wed Jul 20 17:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:43:55.003536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e442-ff59-44a6-b080-c6fc431ebf5d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:55.003982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e442-ff59-44a6-b080-c6fc431ebf5d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:55.004095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e442-ff59-44a6-b080-c6fc431ebf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:55.197056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a1f1e442-ff59-44a6-b080-c6fc431ebf5d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:43:55.198281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a1f1e442-ff59-44a6-b080-c6fc431ebf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:55.292458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 745/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Wed Jul 20 17:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:43:55.332419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702e47c-9be7-48cf-b656-afbc6b0c62ad None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:43:55.333984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6702e47c-9be7-48cf-b656-afbc6b0c62ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:55.435752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702e47c-9be7-48cf-b656-afbc6b0c62ad None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:43:55.506857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe3f6e-28d6-4688-9465-9386d989e20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:43:55.507497 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3f6e-28d6-4688-9465-9386d989e20d service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:43:55.507783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3f6e-28d6-4688-9465-9386d989e20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:43:55.508060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3f6e-28d6-4688-9465-9386d989e20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:43:55.509726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-30fe3f6e-28d6-4688-9465-9386d989e20d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:43:55.518712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:43:55.617974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe3f6e-28d6-4688-9465-9386d989e20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:43:55.619320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe3f6e-28d6-4688-9465-9386d989e20d service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:43:55.637580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 748/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:44:05.290566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=82 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:44:05.635304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=50 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:32.275120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:32.275714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:32.275714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:32.347876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:45:32.349146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:32.451657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 746/1494] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:45:32.459762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6dbac-4bb6-4634-bfa9-b4c38ffb90fb None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:32.460258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6dbac-4bb6-4634-bfa9-b4c38ffb90fb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:32.460365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6dbac-4bb6-4634-bfa9-b4c38ffb90fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:32.532326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b6f6dbac-4bb6-4634-bfa9-b4c38ffb90fb None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:45:32.533544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b6f6dbac-4bb6-4634-bfa9-b4c38ffb90fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:32.626884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 749/1495] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:45:32.631396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:32.632616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:32.643710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae081b71-d975-404c-bac5-ec4798a93021 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:32.705909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:32.750587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:32.751245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:32.751523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:32.751797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:32.752556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:45:32.752864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:32.753917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:32.753917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:45:32.753917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:45:32.754792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:32.766216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 747/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:45:32.774998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6dbac-4bb6-4634-bfa9-b4c38ffb90fb None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:32.785648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6dbac-4bb6-4634-bfa9-b4c38ffb90fb None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:32.846734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:32.880865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:32.881443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:32.881722 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:32.881993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:32.882673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:32.884353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING py.warnings [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:32.884353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: warnings.warn(msg) Jul 20 17:45:32.884353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: Jul 20 17:45:32.885554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:33.018429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 750/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:45:32 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:45:33.023261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:33.024867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:33.057978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c4d8b-95dd-4828-84b5-c6ba63f1f80b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:33.159285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:33.160578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:33.161121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:33.161402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:33.162091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:33.163118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:33.163118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:45:33.163118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:45:33.164328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:33.281064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 748/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:45:33 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:45:33.285346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:33.286851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:33.347504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b95c8-8733-42d2-8603-92e4ca37b28b admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:33.427097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:33.437525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:33.438186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:33.438469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:33.438741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:33.439458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:33.440484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING py.warnings [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:33.440484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: warnings.warn(msg) Jul 20 17:45:33.440484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: Jul 20 17:45:33.441883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:33.453822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 751/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:45:33.458494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:33.460041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:33.471871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4338c0-2134-4d46-be4a-5cdf4c63d13b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:33.496112 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:33.564557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da20ff-d1ef-49f0-9cf8-14ce05e28ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:33.565217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da20ff-d1ef-49f0-9cf8-14ce05e28ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:33.565496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da20ff-d1ef-49f0-9cf8-14ce05e28ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:33.565769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da20ff-d1ef-49f0-9cf8-14ce05e28ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:33.567075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da20ff-d1ef-49f0-9cf8-14ce05e28ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:33.568074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-11da20ff-d1ef-49f0-9cf8-14ce05e28ae1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:33.568074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:45:33.568074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:45:33.568973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da20ff-d1ef-49f0-9cf8-14ce05e28ae1 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:33.580888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 749/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:45:33.585000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:33.586517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:33.619249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e59e3-7ef6-4f8c-9f35-c2e91d190839 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:33.712022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f87df8-a294-4d52-8aa6-6ecceb66ac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:33.712866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87df8-a294-4d52-8aa6-6ecceb66ac50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:33.713155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87df8-a294-4d52-8aa6-6ecceb66ac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:33.713433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87df8-a294-4d52-8aa6-6ecceb66ac50 admin admin] PATH_INFO: `/v3/projects/a94cb04296764abdbcedddcbd74a66f3/users/b2f1640201eb4f3db3f10ea7f2793e6c/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:33.734786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f87df8-a294-4d52-8aa6-6ecceb66ac50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94cb04296764abdbcedddcbd74a66f3, user_id=b2f1640201eb4f3db3f10ea7f2793e6c, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:33.735807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING py.warnings [None req-c5f87df8-a294-4d52-8aa6-6ecceb66ac50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:33.735807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: warnings.warn(msg) Jul 20 17:45:33.735807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: Jul 20 17:45:33.737841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f87df8-a294-4d52-8aa6-6ecceb66ac50 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:33.835250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 752/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:45:33 2022] PUT /identity/v3/projects/a94cb04296764abdbcedddcbd74a66f3/users/b2f1640201eb4f3db3f10ea7f2793e6c/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:45:33.847149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b8b3f-bfba-4e85-9ea5-7b6f0257a86a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:33.847440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b8b3f-bfba-4e85-9ea5-7b6f0257a86a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:33.847719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b8b3f-bfba-4e85-9ea5-7b6f0257a86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:33.901934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-692b8b3f-bfba-4e85-9ea5-7b6f0257a86a None None] MFA Rules not processed for user `b2f1640201eb4f3db3f10ea7f2793e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:45:33.903138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-692b8b3f-bfba-4e85-9ea5-7b6f0257a86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:33.978232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 750/1502] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:45:33.991947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b66c-5b05-45be-82d5-241351981e27 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:33.992241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b66c-5b05-45be-82d5-241351981e27 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:33.992555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b66c-5b05-45be-82d5-241351981e27 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:33.993921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 753/1503] 10.222.0.42 () {54 vars in 964 bytes} [Wed Jul 20 17:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:45:33.998760 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97a60-0b29-4c89-982b-936b1de09d01 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:33.999059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97a60-0b29-4c89-982b-936b1de09d01 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:33.999342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97a60-0b29-4c89-982b-936b1de09d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:34.028098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:34.073008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-33d97a60-0b29-4c89-982b-936b1de09d01 None None] MFA Rules not processed for user `9706ef808a224c83aca165c595b596ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:45:34.074216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-33d97a60-0b29-4c89-982b-936b1de09d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:34.150698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 751/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Jul 20 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:45:34.158392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b66c-5b05-45be-82d5-241351981e27 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:34.159908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5154b66c-5b05-45be-82d5-241351981e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:34.209619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:34.261661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b66c-5b05-45be-82d5-241351981e27 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:34.336967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:34.337938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:34.338222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:34.338501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:34.340588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:34.437893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:34.439334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:34.442938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:34.458083 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 754/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:45:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:45:34.561658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a083-70e4-43dd-a987-0effa32762c2 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:34.562092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a083-70e4-43dd-a987-0effa32762c2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:34.562212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a083-70e4-43dd-a987-0effa32762c2 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:34.563499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 752/1506] 10.222.0.42 () {52 vars in 829 bytes} [Wed Jul 20 17:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:45:34.567265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:34.568495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:34.663633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d646da-1e84-44a4-b23e-25eb56b45f0a service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:34.756689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:34.757551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:34.757827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:34.758094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] PATH_INFO: `/v3/users/b2f1640201eb4f3db3f10ea7f2793e6c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:34.770096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f1640201eb4f3db3f10ea7f2793e6c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:34.771170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING py.warnings [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:34.771170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: warnings.warn(msg) Jul 20 17:45:34.771170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: Jul 20 17:45:34.772362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:34.782009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:35.023862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f1640201eb4f3db3f10ea7f2793e6c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:35.024405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f1640201eb4f3db3f10ea7f2793e6c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:35.024862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f1640201eb4f3db3f10ea7f2793e6c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:35.036990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f1640201eb4f3db3f10ea7f2793e6c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:35.166206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f1640201eb4f3db3f10ea7f2793e6c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:35.190547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 755/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:45:34 2022] DELETE /identity/v3/users/b2f1640201eb4f3db3f10ea7f2793e6c => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:45:35.195646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:35.195937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:35.196218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:35.250028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:45:35.251241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:35.325470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:35.337319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 753/1508] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:45:35.346682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:35.347935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:35.446849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9333951-ce8c-40e7-b004-cd2de14e8121 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:35.500973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:35.519502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baadf1-356c-4599-af5f-1b700fd6cf32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:35.520142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baadf1-356c-4599-af5f-1b700fd6cf32 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:35.520448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baadf1-356c-4599-af5f-1b700fd6cf32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:35.520735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baadf1-356c-4599-af5f-1b700fd6cf32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:35.522361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-85baadf1-356c-4599-af5f-1b700fd6cf32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:35.629687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baadf1-356c-4599-af5f-1b700fd6cf32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:35.631062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baadf1-356c-4599-af5f-1b700fd6cf32 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:35.648753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 756/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:45:36.196613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:45:36.197874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:45:36.209910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c66b6-f0b4-4a85-999b-362ab5624d49 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:45:36.306380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:45:36.307241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:45:36.307526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:45:36.307804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] PATH_INFO: `/v3/projects/a94cb04296764abdbcedddcbd74a66f3` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:45:36.318242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94cb04296764abdbcedddcbd74a66f3)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:45:36.319271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:45:36.319271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:45:36.319271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:45:36.320500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:45:36.456191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94cb04296764abdbcedddcbd74a66f3', 'request_id': 'req-54486569-9064-40f7-b0cc-7a880e4f3196'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:36.462847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94cb04296764abdbcedddcbd74a66f3', 'request_id': 'req-54486569-9064-40f7-b0cc-7a880e4f3196'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:36.477194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94cb04296764abdbcedddcbd74a66f3', 'request_id': 'req-54486569-9064-40f7-b0cc-7a880e4f3196'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:36.617166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] The token cache is being invalidate because project a94cb04296764abdbcedddcbd74a66f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:45:36.617456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:36.617973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-54486569-9064-40f7-b0cc-7a880e4f3196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:45:36.619376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 754/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:45:36 2022] DELETE /identity/v3/projects/a94cb04296764abdbcedddcbd74a66f3 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:45:45.646453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=62 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:45:46.614384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:47:20.799030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6890b-6a1a-4932-b7c1-374df43278f3 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:20.799629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6890b-6a1a-4932-b7c1-374df43278f3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:20.799629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6890b-6a1a-4932-b7c1-374df43278f3 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:20.800997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:20.806217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341827cb-8fae-4599-82c1-8473daa52950 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:20.806635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341827cb-8fae-4599-82c1-8473daa52950 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:20.806783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341827cb-8fae-4599-82c1-8473daa52950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:20.884976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-341827cb-8fae-4599-82c1-8473daa52950 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:20.886441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-341827cb-8fae-4599-82c1-8473daa52950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:20.991024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:21.144026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6890b-6a1a-4932-b7c1-374df43278f3 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:21.146166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-add6890b-6a1a-4932-b7c1-374df43278f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:21.266211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6890b-6a1a-4932-b7c1-374df43278f3 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:21.272119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facea3ac-3a96-48d3-9336-ea8f27064432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:21.272786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facea3ac-3a96-48d3-9336-ea8f27064432 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:21.273073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facea3ac-3a96-48d3-9336-ea8f27064432 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:21.273349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facea3ac-3a96-48d3-9336-ea8f27064432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:21.281225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facea3ac-3a96-48d3-9336-ea8f27064432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:21.282795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facea3ac-3a96-48d3-9336-ea8f27064432 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:21.291948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 758/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:22.109039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df95314-250b-4920-8493-b9315d5a6107 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:22.109513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df95314-250b-4920-8493-b9315d5a6107 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:22.109620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df95314-250b-4920-8493-b9315d5a6107 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:22.111605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:22.116400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb9619-e92e-47c7-8f7d-5f78e8a7a0ea None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:22.116843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb9619-e92e-47c7-8f7d-5f78e8a7a0ea None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:22.117004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb9619-e92e-47c7-8f7d-5f78e8a7a0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:22.136110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f3cb9619-e92e-47c7-8f7d-5f78e8a7a0ea None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:22.137292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f3cb9619-e92e-47c7-8f7d-5f78e8a7a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:22.145228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:22.295761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df95314-250b-4920-8493-b9315d5a6107 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:22.310958 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df95314-250b-4920-8493-b9315d5a6107 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:22.316846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ae781-d7c8-40eb-8273-531c173e6df2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:22.317475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ae781-d7c8-40eb-8273-531c173e6df2 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:22.317758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ae781-d7c8-40eb-8273-531c173e6df2 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:22.318034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ae781-d7c8-40eb-8273-531c173e6df2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:22.325940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ae781-d7c8-40eb-8273-531c173e6df2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:22.327518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ae781-d7c8-40eb-8273-531c173e6df2 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:22.337624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 757/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:22.489860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa056a-cdc7-40d2-a875-c9dfda188e66 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:22.490131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa056a-cdc7-40d2-a875-c9dfda188e66 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:22.490409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa056a-cdc7-40d2-a875-c9dfda188e66 None None] PATH_INFO: `/v3` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:22.491739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 760/1517] 10.222.0.18 () {54 vars in 925 bytes} [Wed Jul 20 17:47:22 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:47:22.497966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186378ae-98e9-42ef-b4d7-990b6e17dca9 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:22.498261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186378ae-98e9-42ef-b4d7-990b6e17dca9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:22.498541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186378ae-98e9-42ef-b4d7-990b6e17dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:22.557147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-186378ae-98e9-42ef-b4d7-990b6e17dca9 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:22.558354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-186378ae-98e9-42ef-b4d7-990b6e17dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:22.575398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:47:22.629599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 758/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jul 20 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:31.847037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c2ac-cd23-4cb1-bf95-b46ba0ecc12e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:31.847608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c2ac-cd23-4cb1-bf95-b46ba0ecc12e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:31.847708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c2ac-cd23-4cb1-bf95-b46ba0ecc12e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:31.849066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 761/1519] 10.222.0.42 () {54 vars in 961 bytes} [Wed Jul 20 17:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:31.854063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121c3a-c0b3-45fd-a407-d047661d7c5b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:31.854462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121c3a-c0b3-45fd-a407-d047661d7c5b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:31.854616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121c3a-c0b3-45fd-a407-d047661d7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:31.872704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-83121c3a-c0b3-45fd-a407-d047661d7c5b None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:31.873894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-83121c3a-c0b3-45fd-a407-d047661d7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:31.882447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 759/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Jul 20 17:47:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:31.889861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c2ac-cd23-4cb1-bf95-b46ba0ecc12e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:31.906949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c2ac-cd23-4cb1-bf95-b46ba0ecc12e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:31.916739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff47ef-cec6-439b-8457-f2090499e9f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:31.917379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff47ef-cec6-439b-8457-f2090499e9f4 service placement] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:31.917669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff47ef-cec6-439b-8457-f2090499e9f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:31.917947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff47ef-cec6-439b-8457-f2090499e9f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:31.927791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff47ef-cec6-439b-8457-f2090499e9f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:31.929420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff47ef-cec6-439b-8457-f2090499e9f4 service placement] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:31.939186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 762/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:33.526823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30ff1-4cf1-486d-8038-253d95349c3a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:33.527431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30ff1-4cf1-486d-8038-253d95349c3a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:33.527542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30ff1-4cf1-486d-8038-253d95349c3a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:33.528944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:33.534331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4b40-13ca-4e96-94f7-42454ab40b04 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:33.534769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4b40-13ca-4e96-94f7-42454ab40b04 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:33.534884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4b40-13ca-4e96-94f7-42454ab40b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:33.553053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d5ae4b40-13ca-4e96-94f7-42454ab40b04 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:33.554245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d5ae4b40-13ca-4e96-94f7-42454ab40b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:33.562232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:33.718438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30ff1-4cf1-486d-8038-253d95349c3a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:33.732986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30ff1-4cf1-486d-8038-253d95349c3a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:33.738889 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf91ec9-5760-44a2-8200-5e6c28f8b4d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:33.739524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91ec9-5760-44a2-8200-5e6c28f8b4d6 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:33.739808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91ec9-5760-44a2-8200-5e6c28f8b4d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:33.740085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91ec9-5760-44a2-8200-5e6c28f8b4d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:33.748195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf91ec9-5760-44a2-8200-5e6c28f8b4d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:33.749806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf91ec9-5760-44a2-8200-5e6c28f8b4d6 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:33.759693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 761/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:34.581989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb0f1d-d235-472a-9486-24361cfa1e17 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:34.582550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb0f1d-d235-472a-9486-24361cfa1e17 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:34.582550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb0f1d-d235-472a-9486-24361cfa1e17 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:34.583910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 17:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:34.588837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bcb7f-a9bb-4c24-a34b-9b94f2518664 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:34.589269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bcb7f-a9bb-4c24-a34b-9b94f2518664 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:34.589401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bcb7f-a9bb-4c24-a34b-9b94f2518664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:34.607433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7d3bcb7f-a9bb-4c24-a34b-9b94f2518664 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:34.608673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7d3bcb7f-a9bb-4c24-a34b-9b94f2518664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:34.616443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 17:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:34.780412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb0f1d-d235-472a-9486-24361cfa1e17 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:34.796095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb0f1d-d235-472a-9486-24361cfa1e17 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:34.801557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175841b-13f1-48c3-bd88-c3f0b8186ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:34.802188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175841b-13f1-48c3-bd88-c3f0b8186ed4 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:34.802471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175841b-13f1-48c3-bd88-c3f0b8186ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:34.802748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175841b-13f1-48c3-bd88-c3f0b8186ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:34.811064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175841b-13f1-48c3-bd88-c3f0b8186ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:34.812711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175841b-13f1-48c3-bd88-c3f0b8186ed4 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:34.822428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 765/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:38.747955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0e2a-3a5a-4f76-8c27-695e8c280bf3 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:38.748545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0e2a-3a5a-4f76-8c27-695e8c280bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:38.748545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0e2a-3a5a-4f76-8c27-695e8c280bf3 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:38.749934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:38.755581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf135733-c189-4caa-b6b6-ac432c651b9e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:38.755987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf135733-c189-4caa-b6b6-ac432c651b9e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:38.756134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf135733-c189-4caa-b6b6-ac432c651b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:38.774659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-bf135733-c189-4caa-b6b6-ac432c651b9e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:38.775848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bf135733-c189-4caa-b6b6-ac432c651b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:38.783536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:38.856104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa0e2a-3a5a-4f76-8c27-695e8c280bf3 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:38.857842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-36fa0e2a-3a5a-4f76-8c27-695e8c280bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:38.934550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa0e2a-3a5a-4f76-8c27-695e8c280bf3 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:38.938092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7e76e-d49c-4d42-bc02-e2ee1310b876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:38.938732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e76e-d49c-4d42-bc02-e2ee1310b876 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:38.939015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e76e-d49c-4d42-bc02-e2ee1310b876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:38.939293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7e76e-d49c-4d42-bc02-e2ee1310b876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:38.948041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7e76e-d49c-4d42-bc02-e2ee1310b876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:38.949510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7e76e-d49c-4d42-bc02-e2ee1310b876 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:38.964128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 764/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:41.002115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968183cd-6564-4754-b1b6-de31603ca996 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:41.003178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968183cd-6564-4754-b1b6-de31603ca996 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:41.003178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968183cd-6564-4754-b1b6-de31603ca996 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:41.004598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:47:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:47:41.009813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1f06-4b11-408c-9e0f-8b6461863270 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:41.018633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1f06-4b11-408c-9e0f-8b6461863270 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:41.018633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1f06-4b11-408c-9e0f-8b6461863270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:41.028821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e4bd1f06-4b11-408c-9e0f-8b6461863270 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:41.030008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e4bd1f06-4b11-408c-9e0f-8b6461863270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:41.037666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:41.075316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968183cd-6564-4754-b1b6-de31603ca996 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:41.085997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968183cd-6564-4754-b1b6-de31603ca996 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:41.089687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f213d3a-bac9-4b94-8b77-dea8a01ed94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:41.090333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f213d3a-bac9-4b94-8b77-dea8a01ed94a service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:41.090616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f213d3a-bac9-4b94-8b77-dea8a01ed94a service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:41.090915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f213d3a-bac9-4b94-8b77-dea8a01ed94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:41.100379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f213d3a-bac9-4b94-8b77-dea8a01ed94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:41.101886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f213d3a-bac9-4b94-8b77-dea8a01ed94a service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:41.112140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 768/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:44.690552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022aa35-4529-4429-846d-26ba19ed4ca5 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:44.691111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022aa35-4529-4429-846d-26ba19ed4ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:44.691111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022aa35-4529-4429-846d-26ba19ed4ca5 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:44.692503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:44.697675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6d6e-2537-4a18-a8f8-f88e91b051c5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:44.698090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6d6e-2537-4a18-a8f8-f88e91b051c5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:44.698241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6d6e-2537-4a18-a8f8-f88e91b051c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:44.715986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-28eb6d6e-2537-4a18-a8f8-f88e91b051c5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:44.717212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-28eb6d6e-2537-4a18-a8f8-f88e91b051c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:44.725510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:45.032287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022aa35-4529-4429-846d-26ba19ed4ca5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:45.034550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9022aa35-4529-4429-846d-26ba19ed4ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:45.099442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:47:45.112501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022aa35-4529-4429-846d-26ba19ed4ca5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:45.116094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb8f38-4039-4168-a50e-f4a3cc1477bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:45.116782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb8f38-4039-4168-a50e-f4a3cc1477bd service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:45.117061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb8f38-4039-4168-a50e-f4a3cc1477bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:45.117343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb8f38-4039-4168-a50e-f4a3cc1477bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:45.126399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb8f38-4039-4168-a50e-f4a3cc1477bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:45.127874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb8f38-4039-4168-a50e-f4a3cc1477bd service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:45.137298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 767/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:47.214107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f9252-b2cc-48ae-a5a8-4f35a45b0ae8 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:47.214649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f9252-b2cc-48ae-a5a8-4f35a45b0ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:47.214649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f9252-b2cc-48ae-a5a8-4f35a45b0ae8 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:47.216128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 17:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:47:47.221977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d56c2-6adb-4cac-9a82-4c3c3abdea1b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:47.222385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d56c2-6adb-4cac-9a82-4c3c3abdea1b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:47.222538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d56c2-6adb-4cac-9a82-4c3c3abdea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:47.240156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3c7d56c2-6adb-4cac-9a82-4c3c3abdea1b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:47:47.241388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3c7d56c2-6adb-4cac-9a82-4c3c3abdea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:47.249121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 17:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:47:47.550716 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f9252-b2cc-48ae-a5a8-4f35a45b0ae8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:47:47.552516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-362f9252-b2cc-48ae-a5a8-4f35a45b0ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:47:47.560234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=84 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:47:47.566407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f9252-b2cc-48ae-a5a8-4f35a45b0ae8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:47:47.569915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924d7a-68c5-42dc-b158-a87b8351a0f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:47:47.572383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924d7a-68c5-42dc-b158-a87b8351a0f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:47:47.572383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924d7a-68c5-42dc-b158-a87b8351a0f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:47:47.572383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924d7a-68c5-42dc-b158-a87b8351a0f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:47:47.579790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924d7a-68c5-42dc-b158-a87b8351a0f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:47:47.581794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924d7a-68c5-42dc-b158-a87b8351a0f4 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:47:47.591224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 771/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:47:57.235488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=11 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:47:57.592119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:16.781557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc341b-9949-43a2-b1b1-5a92e32eaad5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:16.782209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc341b-9949-43a2-b1b1-5a92e32eaad5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:16.782209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc341b-9949-43a2-b1b1-5a92e32eaad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:16.803937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-13bc341b-9949-43a2-b1b1-5a92e32eaad5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:16.805169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-13bc341b-9949-43a2-b1b1-5a92e32eaad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:16.821439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 769/1540] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:16.830506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d535b-0e79-451c-b773-98e365c420e0 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:16.830974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d535b-0e79-451c-b773-98e365c420e0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:16.831092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d535b-0e79-451c-b773-98e365c420e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:16.862185 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ce3d535b-0e79-451c-b773-98e365c420e0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:16.863563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ce3d535b-0e79-451c-b773-98e365c420e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:16.874272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 772/1541] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:49:16.879087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc341b-9949-43a2-b1b1-5a92e32eaad5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:16.894628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc341b-9949-43a2-b1b1-5a92e32eaad5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:16.901926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:16.903198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:16.903484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:16.903773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:16.904552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:16.904865 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:16.907271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:16.925345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 770/1542] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:16.943011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d535b-0e79-451c-b773-98e365c420e0 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:16.953794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d535b-0e79-451c-b773-98e365c420e0 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:16.962836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa301-196c-4767-9a6e-5b67b36ec796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:16.964269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa301-196c-4767-9a6e-5b67b36ec796 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:16.964589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa301-196c-4767-9a6e-5b67b36ec796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:16.964872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa301-196c-4767-9a6e-5b67b36ec796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:16.965583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa301-196c-4767-9a6e-5b67b36ec796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:16.967943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa301-196c-4767-9a6e-5b67b36ec796 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.068587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 773/1543] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:16 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:17.073405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.088467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fd974-8fc1-4c7a-9699-28b212842469 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.153589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa8bdb-05e3-4252-b1ed-aab163e36d0b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.153955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa8bdb-05e3-4252-b1ed-aab163e36d0b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.154159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa8bdb-05e3-4252-b1ed-aab163e36d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.204542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef9550-2e5c-4f40-a3e9-deea6f593386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.210088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef9550-2e5c-4f40-a3e9-deea6f593386 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.210204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e6aa8bdb-05e3-4252-b1ed-aab163e36d0b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.210204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e6aa8bdb-05e3-4252-b1ed-aab163e36d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.210370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef9550-2e5c-4f40-a3e9-deea6f593386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.210370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef9550-2e5c-4f40-a3e9-deea6f593386 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.210370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef9550-2e5c-4f40-a3e9-deea6f593386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.217379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef9550-2e5c-4f40-a3e9-deea6f593386 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.233504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 774/1544] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:49:17.254922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad710a-8e43-4f1b-8c4b-61298ceb1e40 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.255210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad710a-8e43-4f1b-8c4b-61298ceb1e40 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.255489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad710a-8e43-4f1b-8c4b-61298ceb1e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.282385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-87ad710a-8e43-4f1b-8c4b-61298ceb1e40 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.283816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-87ad710a-8e43-4f1b-8c4b-61298ceb1e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.297865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 775/1545] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:49:17.299265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ced49f-0351-4686-ac31-28150da189f5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.301859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ced49f-0351-4686-ac31-28150da189f5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.301859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ced49f-0351-4686-ac31-28150da189f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.330348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c0ced49f-0351-4686-ac31-28150da189f5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.331768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c0ced49f-0351-4686-ac31-28150da189f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.340555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 776/1546] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.341401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ced49f-0351-4686-ac31-28150da189f5 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.352260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ced49f-0351-4686-ac31-28150da189f5 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.363117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.363789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.364080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.364359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.365180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:17.368025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.373522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 771/1547] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/users => generated 295 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:17.374652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c908-fec2-4da5-a777-d29986aa3546 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.374942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c908-fec2-4da5-a777-d29986aa3546 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.375281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75c908-fec2-4da5-a777-d29986aa3546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.377443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING py.warnings [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:49:17.377443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: warnings.warn(msg) Jul 20 17:49:17.377443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: Jul 20 17:49:17.385454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.414504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 777/1548] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:17.416257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.429078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5e75c908-fec2-4da5-a777-d29986aa3546 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.430302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5e75c908-fec2-4da5-a777-d29986aa3546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.447791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb8f10-43a8-4f69-8b82-3ffab0a814e4 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.455212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 772/1549] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.457772 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c908-fec2-4da5-a777-d29986aa3546 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.458743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.459426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.459709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.459987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.466763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.468389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.487096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75c908-fec2-4da5-a777-d29986aa3546 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.496051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 778/1550] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:17.497908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.499935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181710e-46b8-4f97-86b4-3b9484ba2d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.501171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181710e-46b8-4f97-86b4-3b9484ba2d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.501454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181710e-46b8-4f97-86b4-3b9484ba2d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.501728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181710e-46b8-4f97-86b4-3b9484ba2d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.502429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181710e-46b8-4f97-86b4-3b9484ba2d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.503464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-5181710e-46b8-4f97-86b4-3b9484ba2d13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:49:17.503464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:49:17.503464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:49:17.504686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181710e-46b8-4f97-86b4-3b9484ba2d13 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.525346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dba35b-4b86-4516-9c87-5312def925f7 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.534476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139466-10be-4d23-833a-415186333332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.535155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139466-10be-4d23-833a-415186333332 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.535441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139466-10be-4d23-833a-415186333332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.535720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139466-10be-4d23-833a-415186333332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.536475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6139466-10be-4d23-833a-415186333332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:17.536791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6139466-10be-4d23-833a-415186333332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.538880 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6139466-10be-4d23-833a-415186333332 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.553459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 779/1551] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:17.554685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139466-10be-4d23-833a-415186333332 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.567272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139466-10be-4d23-833a-415186333332 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.572467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692838c-eae4-43a9-b839-15fa4ddc1182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.573223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692838c-eae4-43a9-b839-15fa4ddc1182 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.573585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692838c-eae4-43a9-b839-15fa4ddc1182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.573940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692838c-eae4-43a9-b839-15fa4ddc1182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.574940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692838c-eae4-43a9-b839-15fa4ddc1182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.576739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692838c-eae4-43a9-b839-15fa4ddc1182 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.587465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 780/1552] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:17.588730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e81a-e223-4f19-af99-02975136b018 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.589090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e81a-e223-4f19-af99-02975136b018 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.589443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e81a-e223-4f19-af99-02975136b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.607425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6426e81a-e223-4f19-af99-02975136b018 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.611707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6426e81a-e223-4f19-af99-02975136b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.617173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 781/1553] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.618481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e539-12e3-4095-a85e-d682fc57bbd0 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.619000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e539-12e3-4095-a85e-d682fc57bbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.619384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0e539-12e3-4095-a85e-d682fc57bbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.635130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 773/1554] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:17.638575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434db35-3474-45cd-a955-1f675bb1bfbe None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.638575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434db35-3474-45cd-a955-1f675bb1bfbe None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.638575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434db35-3474-45cd-a955-1f675bb1bfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.639700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e1b0e539-12e3-4095-a85e-d682fc57bbd0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.641547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e1b0e539-12e3-4095-a85e-d682fc57bbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.661695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7434db35-3474-45cd-a955-1f675bb1bfbe None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.662908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7434db35-3474-45cd-a955-1f675bb1bfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.704830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 782/1555] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.704830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0e539-12e3-4095-a85e-d682fc57bbd0 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.712410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0e539-12e3-4095-a85e-d682fc57bbd0 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.717138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc1010-15c9-4852-9c34-10aef4d97b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.717780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1010-15c9-4852-9c34-10aef4d97b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.718068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1010-15c9-4852-9c34-10aef4d97b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.718349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1010-15c9-4852-9c34-10aef4d97b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.719567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc1010-15c9-4852-9c34-10aef4d97b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.721508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc1010-15c9-4852-9c34-10aef4d97b09 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.724025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 774/1556] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.725316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3efc64-f362-46a9-971d-9d0771c78ba7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.725611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3efc64-f362-46a9-971d-9d0771c78ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.725892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3efc64-f362-46a9-971d-9d0771c78ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.743061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3f3efc64-f362-46a9-971d-9d0771c78ba7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.744256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3f3efc64-f362-46a9-971d-9d0771c78ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.752817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 775/1557] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.755123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3efc64-f362-46a9-971d-9d0771c78ba7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.763120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3efc64-f362-46a9-971d-9d0771c78ba7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.768398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.769293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.769581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.769861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] PATH_INFO: `/v3/projects/1e0c8bf2bfe34311961a4bde7f102944/users/311c19d7062648f78ba4c2090c58f14d/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.784149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0c8bf2bfe34311961a4bde7f102944, user_id=311c19d7062648f78ba4c2090c58f14d, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.785223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:49:17.785223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:49:17.785223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:49:17.787168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.808421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 783/1558] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:17.809865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea222b-76b8-4904-9a53-a071e5342901 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.810162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea222b-76b8-4904-9a53-a071e5342901 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.810441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea222b-76b8-4904-9a53-a071e5342901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.829317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c5ea222b-76b8-4904-9a53-a071e5342901 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.830510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c5ea222b-76b8-4904-9a53-a071e5342901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.852013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:17.873849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 776/1559] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:17 2022] PUT /identity/v3/projects/1e0c8bf2bfe34311961a4bde7f102944/users/311c19d7062648f78ba4c2090c58f14d/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:17.874777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:17.887415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:17.892032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 784/1560] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.893330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582624-5c01-49ef-bfb6-d1a0f97837d2 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.893692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582624-5c01-49ef-bfb6-d1a0f97837d2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.894042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582624-5c01-49ef-bfb6-d1a0f97837d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.912021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-00582624-5c01-49ef-bfb6-d1a0f97837d2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:17.913247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-00582624-5c01-49ef-bfb6-d1a0f97837d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:17.943447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46f5e1-979f-4324-a21c-9508ebbd8ad0 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:17.948854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:17.949556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.949843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.950162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.950820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:17.952837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:17.977545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 785/1561] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:17.978939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b2c9-54c4-4fa8-9fb1-79ac6c6e7ddd None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:17.979231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b2c9-54c4-4fa8-9fb1-79ac6c6e7ddd None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:17.979567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b2c9-54c4-4fa8-9fb1-79ac6c6e7ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:17.997275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ee68b2c9-54c4-4fa8-9fb1-79ac6c6e7ddd None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:18.001416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ee68b2c9-54c4-4fa8-9fb1-79ac6c6e7ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:18.006790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 786/1562] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:18.008041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b0e0d-b650-43ea-a0f0-5a87c75b8894 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.008334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b0e0d-b650-43ea-a0f0-5a87c75b8894 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.008644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b0e0d-b650-43ea-a0f0-5a87c75b8894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.025376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c28b0e0d-b650-43ea-a0f0-5a87c75b8894 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:18.026527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c28b0e0d-b650-43ea-a0f0-5a87c75b8894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:18.034225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 787/1563] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:18.035500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229937d-3657-4720-ae83-694457f821ba None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.035795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229937d-3657-4720-ae83-694457f821ba None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.036073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229937d-3657-4720-ae83-694457f821ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.050884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 777/1564] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:17 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.050884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.057425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3229937d-3657-4720-ae83-694457f821ba None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:18.063348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4f3bd-9fa7-4866-b0d4-f1c9bb3b7811 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.063499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3229937d-3657-4720-ae83-694457f821ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:18.064330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.065011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.065301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.065578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.066266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.067125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 788/1565] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:18.068357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e14f50-bb54-4669-8d79-6a6287f7a778 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.068679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e14f50-bb54-4669-8d79-6a6287f7a778 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.068959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e14f50-bb54-4669-8d79-6a6287f7a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.071853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.088730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c2e14f50-bb54-4669-8d79-6a6287f7a778 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:18.089912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c2e14f50-bb54-4669-8d79-6a6287f7a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:18.097552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 789/1566] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:18.100847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e14f50-bb54-4669-8d79-6a6287f7a778 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.109965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e14f50-bb54-4669-8d79-6a6287f7a778 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.113042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cf060-d80c-448e-b8c9-13b844f5aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.113691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cf060-d80c-448e-b8c9-13b844f5aa0e admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.113975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cf060-d80c-448e-b8c9-13b844f5aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.114252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cf060-d80c-448e-b8c9-13b844f5aa0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.114974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cf060-d80c-448e-b8c9-13b844f5aa0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:18.115278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cf060-d80c-448e-b8c9-13b844f5aa0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.117016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cf060-d80c-448e-b8c9-13b844f5aa0e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.129537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 790/1567] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:18.130565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ff0e-4a46-4b3f-be0d-dfa2b2ade2a5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.130855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ff0e-4a46-4b3f-be0d-dfa2b2ade2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.131131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ff0e-4a46-4b3f-be0d-dfa2b2ade2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.148889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-97c4ff0e-4a46-4b3f-be0d-dfa2b2ade2a5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:18.150104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-97c4ff0e-4a46-4b3f-be0d-dfa2b2ade2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:18.158424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 791/1568] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:18.159170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ff0e-4a46-4b3f-be0d-dfa2b2ade2a5 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.169908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ff0e-4a46-4b3f-be0d-dfa2b2ade2a5 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.175274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.175930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.176215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.176519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.177241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:18.177545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.179134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.188798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 778/1569] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.189971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.198403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 792/1570] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:18.198403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.200059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ae128-c2b9-45aa-ae67-ee1abe37dd58 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.204992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.205678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.205960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.206234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.206954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:18.207259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.208203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf97282-6d79-4603-8b8e-585867d77f63 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.208874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.213095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.213899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.214195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.214479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.215208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.216847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.222898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 779/1571] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:18.223678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.230155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 793/1572] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:18.230973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.236763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646484f-b88c-47bc-b727-f38904d79b5e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.241790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.242106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.242387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.242661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.243379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:18.243682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.245292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.258155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 780/1573] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:18.258987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.276934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f33042-d612-4a15-935c-7a98cdc02e2b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.282436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.283154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.283440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.283740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.284448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.286363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.287113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610ec3e-3769-4589-bb4a-913c197c0457 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.290831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.291479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.291786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.292052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.302143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.303287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.308800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:18.314188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 794/1574] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:18.315161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.323950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786520c-02f5-4d18-8651-bda76da0cd4f service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.328536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.329173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.329456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.329733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.330730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:18.330730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.332292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.344705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 795/1575] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:18.345489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.354775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fe997-ea52-428a-99e3-3cf3af505c5d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.359294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.360618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.360848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.361121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.361853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.363406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.373774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 796/1576] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:18.374653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.383710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7546efb-8f57-4fc0-8a96-f2cf74e8529c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.388110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.388861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.389146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.389421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.390144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.392014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.419961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 781/1577] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/projects => generated 403 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.420846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.486816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0f99-98ab-404e-859b-ce0c9b807abe admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.491602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.492273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.492587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.492864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.493590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.495181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.499773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 797/1578] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.501229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.509522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 782/1579] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:18.510359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.519784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7be1b-9064-40f3-ad39-27d76302f68e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.549146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:18.568064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba47e7-eca5-42d1-9b69-bd0d00d07061 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.574088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.574834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.575189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.575535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.576305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.577727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.579349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.579570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.579673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.579673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.580460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.582426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.720746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 798/1581] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/projects => generated 403 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.720954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 783/1581] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/projects => generated 393 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.721053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.722007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.780729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a03c6-6baf-455c-bca8-854928a5219b admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.784667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276dad4-ea9b-46f0-b4af-abad7d100748 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.785917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.786576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.786953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.787155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.787908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.789830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.789975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.790500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.790792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.791072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.791802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.793465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.803667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 784/1582] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:18.804474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.814067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5e230-e713-4862-952c-1b0316ff7dc6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.818705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.819410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.819697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.819980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.820687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.822620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:18.892078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 799/1583] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.892078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.916950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 785/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:18.916950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:18.927367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ddcc7-8807-4a15-ae5d-43cdd50576d4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.929906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:18.971028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0923b-a1be-4443-a488-884b99a5a669 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:18.976024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:18.976902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:18.977199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:18.977477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:18.978172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:18.979189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING py.warnings [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:49:18.979189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: warnings.warn(msg) Jul 20 17:49:18.979189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: Jul 20 17:49:18.980631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.002193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.005634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.005928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.006210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] PATH_INFO: `/v3/projects/6a76562649474d5bb8f326aab70630ab/users/137e21587db048529f771d750da5b208/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.027491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a76562649474d5bb8f326aab70630ab, user_id=137e21587db048529f771d750da5b208, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.032595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.050760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 800/1585] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:18 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:19.051707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.062142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922d7d2-bc19-4b93-89a5-c3a30ced3fa9 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.067101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.067612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.068082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.068900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.069745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.071583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.128062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 786/1586] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:18 2022] PUT /identity/v3/projects/6a76562649474d5bb8f326aab70630ab/users/137e21587db048529f771d750da5b208/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:19.128995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.148896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:19.156839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 801/1587] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:19 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:19.158849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.169469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc182ac9-b5b6-4a2a-9f18-f009f59c382e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.200098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472e013-3fea-4518-9a4b-74a7f87503c8 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.205919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.206608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.206903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.207185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.207898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.209865 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.235915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.236840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.237119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.237390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] PATH_INFO: `/v3/projects/93918dc6b6f445d18a3ecb003524152d/users/3077a7934f9c4407a70a8790b8da9cde/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.253068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93918dc6b6f445d18a3ecb003524152d, user_id=3077a7934f9c4407a70a8790b8da9cde, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.255811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.295746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 787/1588] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:19 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:19.296796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.307717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d696a-8af2-4f46-8284-5e498de570b5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.312707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.313398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.314741 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.314741 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.315112 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.317096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.318884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 802/1589] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:19 2022] PUT /identity/v3/projects/93918dc6b6f445d18a3ecb003524152d/users/3077a7934f9c4407a70a8790b8da9cde/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:19.319744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.388272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6d48d-c2c8-4e4c-84b1-ca35f2f94df0 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.394007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d788de-3c49-41a8-80ee-2c1b0c5105de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.395007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d788de-3c49-41a8-80ee-2c1b0c5105de admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.395302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d788de-3c49-41a8-80ee-2c1b0c5105de admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.395590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d788de-3c49-41a8-80ee-2c1b0c5105de admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.396339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d788de-3c49-41a8-80ee-2c1b0c5105de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.398358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d788de-3c49-41a8-80ee-2c1b0c5105de admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.404384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 788/1590] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:19 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:19.404817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.413599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 803/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.413599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39d6f-1bda-4f6f-b127-fb0e29ce56f9 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.413599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39d6f-1bda-4f6f-b127-fb0e29ce56f9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.413599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39d6f-1bda-4f6f-b127-fb0e29ce56f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.415742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9258c-369c-488b-bf9a-4ac5dbbe6be6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.420460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.421141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.421428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.421704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.422436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.424039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.430538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-cef39d6f-1bda-4f6f-b127-fb0e29ce56f9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:19.433057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-cef39d6f-1bda-4f6f-b127-fb0e29ce56f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:19.434564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 789/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.435373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.440634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 804/1593] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:19.441433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39d6f-1bda-4f6f-b127-fb0e29ce56f9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.447921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad477742-1309-4615-8fe8-66808716d0c9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.450927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39d6f-1bda-4f6f-b127-fb0e29ce56f9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.453086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b1609-3ce2-40a9-95c7-31a32d3b11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.454291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1609-3ce2-40a9-95c7-31a32d3b11cb admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.454572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1609-3ce2-40a9-95c7-31a32d3b11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.454907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1609-3ce2-40a9-95c7-31a32d3b11cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.455579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b1609-3ce2-40a9-95c7-31a32d3b11cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.456053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.456750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.457043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.457240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b1609-3ce2-40a9-95c7-31a32d3b11cb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.457349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.458041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.459638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.461938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:19.468144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 790/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.469498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574faae-558e-47ec-a5f9-e73e7045d69e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.469795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574faae-558e-47ec-a5f9-e73e7045d69e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.470072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574faae-558e-47ec-a5f9-e73e7045d69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.471886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 805/1595] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.471886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.482838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cbe1e-4308-491b-909a-3fcc5940b708 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.487409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8574faae-558e-47ec-a5f9-e73e7045d69e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:19.488304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.488636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8574faae-558e-47ec-a5f9-e73e7045d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:19.489134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.489542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.489825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.490817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.492401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.498146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 791/1596] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:19.498146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574faae-558e-47ec-a5f9-e73e7045d69e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.506606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 806/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.506606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.507845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574faae-558e-47ec-a5f9-e73e7045d69e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.512935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.513605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.513890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.514167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.514893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.515873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d06847-1051-4d05-b410-c45e40b3f012 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.516565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.520680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.521350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.521616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.521894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.522614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.524184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.526976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 792/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.528609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.529874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:19.538218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 807/1599] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.538218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.548612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b856440-4233-4635-a982-4373b27b5308 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.553752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.554392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.557637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.557637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.557637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.557637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.571211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 808/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.571211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.580188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f1f73-3abd-4f38-810c-c1a00b206d38 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.587345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.587345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.587345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.587345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.588241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.589575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.596670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b615e-c180-4d74-a6f5-64522e08b99b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.600386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.601066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.601352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.601691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.603445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 809/1601] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.604285 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.616408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc32af-93a0-4fa0-b8c1-f7addcd166ab admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.619828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.621403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.621554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.622308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.622671 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.623042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.623842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.626152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.633810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 793/1602] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:19.634793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.640102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 810/1603] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:19.640102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.643740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eae103-570c-4ded-b15f-7d67087161d2 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.647771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.653552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.653552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.653552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.653833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27c2de-3b70-4b19-8334-35fec7dc2933 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.655281 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.656759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.657139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.657254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] PATH_INFO: `/v3/projects/5fd29c713a91460b92d3926b0ec56200/users/e2304cd0c41b49c580671cc359b1122d/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.662060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.664072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.674447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd29c713a91460b92d3926b0ec56200, user_id=e2304cd0c41b49c580671cc359b1122d, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.675703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 794/1604] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:19.676556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.677192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.690378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b793b-c84f-42fe-bd95-67f0f43bedae service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.695793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:19.697109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:19.697492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:19.697876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] PATH_INFO: `/v3/projects/bc61628523b44a37b6fc427205136ad9/users/d1d9a99ed3e04d9782a4cfdd41b00376/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:19.717685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc61628523b44a37b6fc427205136ad9, user_id=d1d9a99ed3e04d9782a4cfdd41b00376, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:19.720604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:19.852506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 811/1605] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:19 2022] PUT /identity/v3/projects/5fd29c713a91460b92d3926b0ec56200/users/e2304cd0c41b49c580671cc359b1122d/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:19.852506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.854276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 795/1606] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:19 2022] PUT /identity/v3/projects/bc61628523b44a37b6fc427205136ad9/users/d1d9a99ed3e04d9782a4cfdd41b00376/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:19.857261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:19.889928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:19.947437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b2024-335a-4ea9-bcc7-9e96838b62f3 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:19.999852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab12366-116f-4b9a-8343-6ce4399f8841 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:20.006416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:20.007542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.007886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.008176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] PATH_INFO: `/v3/projects/8eebf9df517849fcbc5a104d28c60257/users/4b000575206c4b98889a8a26c98c5d3a/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.023834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705f92d-8cb1-422d-83b1-c148d94a7919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:20.027936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705f92d-8cb1-422d-83b1-c148d94a7919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.027936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705f92d-8cb1-422d-83b1-c148d94a7919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.027936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705f92d-8cb1-422d-83b1-c148d94a7919 admin admin] PATH_INFO: `/v3/projects/073215a6db1a496385d71ae843886dd4/users/e4dff416fce346b5b7679837bb7a5b5c/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.032657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eebf9df517849fcbc5a104d28c60257, user_id=4b000575206c4b98889a8a26c98c5d3a, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:20.037868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:20.048264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705f92d-8cb1-422d-83b1-c148d94a7919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=073215a6db1a496385d71ae843886dd4, user_id=e4dff416fce346b5b7679837bb7a5b5c, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:20.051243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705f92d-8cb1-422d-83b1-c148d94a7919 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:20.155593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 796/1607] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:19 2022] PUT /identity/v3/projects/8eebf9df517849fcbc5a104d28c60257/users/4b000575206c4b98889a8a26c98c5d3a/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:20.155593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:20.170386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 812/1608] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:19 2022] PUT /identity/v3/projects/073215a6db1a496385d71ae843886dd4/users/e4dff416fce346b5b7679837bb7a5b5c/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:20.171798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331cdff-615f-4f56-9616-4724fb133021 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.172418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331cdff-615f-4f56-9616-4724fb133021 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.172857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331cdff-615f-4f56-9616-4724fb133021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.201932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:20.216632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-a331cdff-615f-4f56-9616-4724fb133021 None None] MFA Rules not processed for user `e2304cd0c41b49c580671cc359b1122d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:20.219621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a331cdff-615f-4f56-9616-4724fb133021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:20.289501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7504e-6a86-4a30-a162-65848936b0ae admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:20.356649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 813/1609] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:20.356649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bc3e-907e-489e-b06d-c594eee6a181 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.356649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bc3e-907e-489e-b06d-c594eee6a181 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.356649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bc3e-907e-489e-b06d-c594eee6a181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.377396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35668de4-fd5a-440d-bda2-a940b11f59ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:20.378421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35668de4-fd5a-440d-bda2-a940b11f59ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.378641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35668de4-fd5a-440d-bda2-a940b11f59ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.378870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35668de4-fd5a-440d-bda2-a940b11f59ff admin admin] PATH_INFO: `/v3/projects/2731e64fe89e484fa08dc52786fe4dd5/users/5c5dba86648f44f3ad71ae095cad4afc/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.404965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35668de4-fd5a-440d-bda2-a940b11f59ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2731e64fe89e484fa08dc52786fe4dd5, user_id=5c5dba86648f44f3ad71ae095cad4afc, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:20.409362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35668de4-fd5a-440d-bda2-a940b11f59ff admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:20.415762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0178bc3e-907e-489e-b06d-c594eee6a181 None None] MFA Rules not processed for user `d1d9a99ed3e04d9782a4cfdd41b00376`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:20.416957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0178bc3e-907e-489e-b06d-c594eee6a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:20.493552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 814/1610] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:20.493552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dadcb-8a29-466b-a3b5-05b91881febe None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.493552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dadcb-8a29-466b-a3b5-05b91881febe None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.493552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dadcb-8a29-466b-a3b5-05b91881febe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.523764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-dc2dadcb-8a29-466b-a3b5-05b91881febe None None] MFA Rules not processed for user `4b000575206c4b98889a8a26c98c5d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:20.529146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-dc2dadcb-8a29-466b-a3b5-05b91881febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:20.559553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 797/1611] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:20 2022] PUT /identity/v3/projects/2731e64fe89e484fa08dc52786fe4dd5/users/5c5dba86648f44f3ad71ae095cad4afc/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:20.566338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b366df-b0c3-4766-b52b-9940e497a506 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.566338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b366df-b0c3-4766-b52b-9940e497a506 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.566338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b366df-b0c3-4766-b52b-9940e497a506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.595417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-70b366df-b0c3-4766-b52b-9940e497a506 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:20.600690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-70b366df-b0c3-4766-b52b-9940e497a506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:20.612580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 815/1612] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:20.613637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dadcb-8a29-466b-a3b5-05b91881febe None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:20.631915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dadcb-8a29-466b-a3b5-05b91881febe None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:20.636510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:20.693732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 798/1613] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:20.693732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b366df-b0c3-4766-b52b-9940e497a506 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:20.710621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31066392-e172-4873-b8cf-642e70e63793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:20.711173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b366df-b0c3-4766-b52b-9940e497a506 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:20.711516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066392-e172-4873-b8cf-642e70e63793 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.711864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066392-e172-4873-b8cf-642e70e63793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.712209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066392-e172-4873-b8cf-642e70e63793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.713063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31066392-e172-4873-b8cf-642e70e63793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:20.715082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31066392-e172-4873-b8cf-642e70e63793 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:20.717127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:20.717808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.718097 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.718603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.720105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:20.722600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:20.835485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 816/1614] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/projects => generated 405 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:20.838633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cb2c2-92d5-4add-bd95-53c896cb5592 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:20.838633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cb2c2-92d5-4add-bd95-53c896cb5592 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:20.838633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cb2c2-92d5-4add-bd95-53c896cb5592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:20.863000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:20.874302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d70cb2c2-92d5-4add-bd95-53c896cb5592 None None] MFA Rules not processed for user `5c5dba86648f44f3ad71ae095cad4afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:20.876365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d70cb2c2-92d5-4add-bd95-53c896cb5592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:20.877031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 799/1615] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/projects => generated 365 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:20.878204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:20.969061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 817/1616] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:20.970603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cb2c2-92d5-4add-bd95-53c896cb5592 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.029687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8bf68-2d55-42c8-8878-de619726e525 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.038126 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.040152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.040687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.041207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.042714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.049849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.051631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cb2c2-92d5-4add-bd95-53c896cb5592 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.057940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.058170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.058456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.058730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.073987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.075993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.090774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 818/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:21.091894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.127663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85dd96-b6e9-4070-a9ca-5bbee1223c00 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.214197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87dbe0-e6b1-4132-99df-977aec8afac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.214197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87dbe0-e6b1-4132-99df-977aec8afac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.214197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87dbe0-e6b1-4132-99df-977aec8afac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.214197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87dbe0-e6b1-4132-99df-977aec8afac8 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.216714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87dbe0-e6b1-4132-99df-977aec8afac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.216714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87dbe0-e6b1-4132-99df-977aec8afac8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.217166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 800/1618] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:20 2022] POST /identity/v3/projects => generated 391 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:21.217166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.272869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5667-b5fb-49dd-a4d4-ee56fdcd6989 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.278898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.280161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.280748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.281125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.282510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.288158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.305541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 819/1619] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:21 2022] POST /identity/v3/users => generated 309 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:21.307555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb791d-46d1-4b1e-98a3-6f729eb25835 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.307992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb791d-46d1-4b1e-98a3-6f729eb25835 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.308416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb791d-46d1-4b1e-98a3-6f729eb25835 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.310491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 820/1620] 10.222.0.42 () {54 vars in 962 bytes} [Wed Jul 20 17:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:21.311386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb791d-46d1-4b1e-98a3-6f729eb25835 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.329275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb791d-46d1-4b1e-98a3-6f729eb25835 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.334748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a950d3-b1ce-465c-9f87-9112722f5a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.335705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a950d3-b1ce-465c-9f87-9112722f5a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.336427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a950d3-b1ce-465c-9f87-9112722f5a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.336938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a950d3-b1ce-465c-9f87-9112722f5a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.337993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a950d3-b1ce-465c-9f87-9112722f5a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.340378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a950d3-b1ce-465c-9f87-9112722f5a49 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.417013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 801/1621] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:21 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:21.418021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.428633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e835397-fed4-4da6-8b09-c1380d6a4a50 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.434120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.434771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.435069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.435345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.436217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.436370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:21.438356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.446435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 821/1622] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:21 2022] POST /identity/v3/users => generated 302 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:21.448875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86a350-49ae-4b27-8006-e2e97bcf3542 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.449336 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86a350-49ae-4b27-8006-e2e97bcf3542 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.449796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86a350-49ae-4b27-8006-e2e97bcf3542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.452831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 802/1623] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:21.453927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.468006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073cf4c-4098-43bb-8f64-a09dc0dd674d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.476413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.476413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.476413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.476413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.476900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.477917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.490541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 803/1624] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:21.490766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.496351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-bc86a350-49ae-4b27-8006-e2e97bcf3542 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:21.498761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bc86a350-49ae-4b27-8006-e2e97bcf3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:21.505955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363862ed-152e-4806-93fe-b8245f6ee84d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.506774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.507417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.507787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.508066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.508834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.510441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.522051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 804/1625] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:21.523322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.535856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1f2e3-70dc-4acf-8075-d26f4e5bf98d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.541457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.542614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.543102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.543566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] PATH_INFO: `/v3/projects/7fddbb32c1f443348ed53bcd633ae412/users/f9815c93377d4a4aaaf02e946083a6d9/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.568716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fddbb32c1f443348ed53bcd633ae412, user_id=f9815c93377d4a4aaaf02e946083a6d9, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.571914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.624633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 822/1626] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Jul 20 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:21.626439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86a350-49ae-4b27-8006-e2e97bcf3542 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.634114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:21.662206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86a350-49ae-4b27-8006-e2e97bcf3542 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.669663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.670673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.671556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.672018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] PATH_INFO: `/v3/projects/467110c550b244de9e996ce21319c333/users/e0d35514496747769a822aa9f1157127/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.692685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467110c550b244de9e996ce21319c333, user_id=e0d35514496747769a822aa9f1157127, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.697186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.740739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 805/1627] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:21 2022] PUT /identity/v3/projects/7fddbb32c1f443348ed53bcd633ae412/users/f9815c93377d4a4aaaf02e946083a6d9/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:21.741461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.837850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3e555-d908-4fed-8f61-428324fe1b8d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:21.842675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:21.843667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:21.844025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:21.844348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] PATH_INFO: `/v3/projects/c3d0a9d63f57472fa588adf60a13ef3a/users/bde3b4f7608c41b6b1f0b7f35541c007/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:21.852409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 823/1628] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:21 2022] PUT /identity/v3/projects/467110c550b244de9e996ce21319c333/users/e0d35514496747769a822aa9f1157127/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:21.852409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:21.862632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d0a9d63f57472fa588adf60a13ef3a, user_id=bde3b4f7608c41b6b1f0b7f35541c007, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:21.866174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:21.975446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b95d7d-83e0-4af3-b705-4170deb759f2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.000569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 806/1629] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:21 2022] PUT /identity/v3/projects/c3d0a9d63f57472fa588adf60a13ef3a/users/bde3b4f7608c41b6b1f0b7f35541c007/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:22.004341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.018610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.022499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.023947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.024197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.149137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf92c6-3bed-48e4-894a-2e1bc69cd36e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.157107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.160144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.166473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.167475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.167972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.172964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.172964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.173219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 824/1630] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:22.173850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.174745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.211229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 807/1631] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:22.212254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.230823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabbcbb-d092-4c13-821d-767c2495940b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.237352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.238456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.244754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.244754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.244754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.244754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.258435 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 808/1632] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:22.259389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.266297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808d8a9-9b9d-4e3e-b004-c6a80db81562 service nova] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.272319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.279942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.279942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.279942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.280357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b5130-c981-4aae-b7a6-2ac827b03d78 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.280478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.280478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.281884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.283075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.283508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.283888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] PATH_INFO: `/v3/projects/7fddbb32c1f443348ed53bcd633ae412/users/f9815c93377d4a4aaaf02e946083a6d9/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.286987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fddbb32c1f443348ed53bcd633ae412, user_id=f9815c93377d4a4aaaf02e946083a6d9, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.290334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.290754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 825/1633] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:22.291764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.304398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23847a12-8426-4a81-baf7-35b91f74fbdb admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.312480 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 809/1634] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:22 2022] PUT /identity/v3/projects/7fddbb32c1f443348ed53bcd633ae412/users/f9815c93377d4a4aaaf02e946083a6d9/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:22.313241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.391149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f08d0f-dcc5-4ca1-863b-f5de6b5ad4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.392210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f08d0f-dcc5-4ca1-863b-f5de6b5ad4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.392626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f08d0f-dcc5-4ca1-863b-f5de6b5ad4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.393248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f08d0f-dcc5-4ca1-863b-f5de6b5ad4b4 admin admin] PATH_INFO: `/v3/projects/c3d0a9d63f57472fa588adf60a13ef3a/users/bde3b4f7608c41b6b1f0b7f35541c007/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.396680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f08d0f-dcc5-4ca1-863b-f5de6b5ad4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d0a9d63f57472fa588adf60a13ef3a, user_id=bde3b4f7608c41b6b1f0b7f35541c007, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.398533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1ccc5-ba06-467c-b005-3fac1979e0e9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.399941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f08d0f-dcc5-4ca1-863b-f5de6b5ad4b4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.404848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a654843-c150-4e4c-980d-28d273da2653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.406611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a654843-c150-4e4c-980d-28d273da2653 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.406901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a654843-c150-4e4c-980d-28d273da2653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.407186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a654843-c150-4e4c-980d-28d273da2653 admin admin] PATH_INFO: `/v3/projects/467110c550b244de9e996ce21319c333/users/e0d35514496747769a822aa9f1157127/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.409631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a654843-c150-4e4c-980d-28d273da2653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467110c550b244de9e996ce21319c333, user_id=e0d35514496747769a822aa9f1157127, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.412664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a654843-c150-4e4c-980d-28d273da2653 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.413625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 826/1635] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:22 2022] PUT /identity/v3/projects/c3d0a9d63f57472fa588adf60a13ef3a/users/bde3b4f7608c41b6b1f0b7f35541c007/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:22.415093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc624319-3281-45c0-b4fe-999e5c7b6b89 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.415487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc624319-3281-45c0-b4fe-999e5c7b6b89 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.415876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc624319-3281-45c0-b4fe-999e5c7b6b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.441769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 810/1636] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:22 2022] PUT /identity/v3/projects/467110c550b244de9e996ce21319c333/users/e0d35514496747769a822aa9f1157127/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:22.443391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052516a-816d-41c0-b6ad-88fdb1c07c7f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.446121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052516a-816d-41c0-b6ad-88fdb1c07c7f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.446388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052516a-816d-41c0-b6ad-88fdb1c07c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.464783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-cc624319-3281-45c0-b4fe-999e5c7b6b89 None None] MFA Rules not processed for user `f9815c93377d4a4aaaf02e946083a6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:22.466812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-cc624319-3281-45c0-b4fe-999e5c7b6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:22.489231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7052516a-816d-41c0-b6ad-88fdb1c07c7f None None] MFA Rules not processed for user `bde3b4f7608c41b6b1f0b7f35541c007`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:22.491195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7052516a-816d-41c0-b6ad-88fdb1c07c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:22.566264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 827/1637] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:22.568599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bdff5-4854-45c6-9dbe-df11ed51a4aa None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.569157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bdff5-4854-45c6-9dbe-df11ed51a4aa None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.569613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bdff5-4854-45c6-9dbe-df11ed51a4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.605590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 811/1638] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:22.606640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052516a-816d-41c0-b6ad-88fdb1c07c7f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.623236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-343bdff5-4854-45c6-9dbe-df11ed51a4aa None None] MFA Rules not processed for user `e0d35514496747769a822aa9f1157127`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:22.625677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-343bdff5-4854-45c6-9dbe-df11ed51a4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:22.670594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052516a-816d-41c0-b6ad-88fdb1c07c7f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.675208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.675873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.676159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.676460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.689016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.691083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.703085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 812/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:22 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:22.704257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.714493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94a9da-722d-40d5-913a-5ad676469906 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:22.720721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7707f6-960b-423e-8f6b-3fdfc42953b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:22.721646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7707f6-960b-423e-8f6b-3fdfc42953b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.722241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7707f6-960b-423e-8f6b-3fdfc42953b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.722707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7707f6-960b-423e-8f6b-3fdfc42953b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.735437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7707f6-960b-423e-8f6b-3fdfc42953b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:22.737744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7707f6-960b-423e-8f6b-3fdfc42953b8 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:22.751272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 813/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:22.754882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec077d1-fbf9-41b7-886d-3845e52b975a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.758176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec077d1-fbf9-41b7-886d-3845e52b975a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.758176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec077d1-fbf9-41b7-886d-3845e52b975a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.771120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 814/1641] 10.222.0.42 () {54 vars in 912 bytes} [Wed Jul 20 17:49:22 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:22.772935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b714b0a-ab81-4c66-8d36-57e4c33b4a2d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:22.773432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b714b0a-ab81-4c66-8d36-57e4c33b4a2d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:22.773941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b714b0a-ab81-4c66-8d36-57e4c33b4a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:22.819202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 828/1642] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:22.827404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bdff5-4854-45c6-9dbe-df11ed51a4aa None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:22.829168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7b714b0a-ab81-4c66-8d36-57e4c33b4a2d None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:22.832876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7b714b0a-ab81-4c66-8d36-57e4c33b4a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:23.104158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bdff5-4854-45c6-9dbe-df11ed51a4aa None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:23.104158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:23.104158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:23.104158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:23.106373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:23.106373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:23.106373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:23.106373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:23.106373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 829/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:49:23.108223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 815/1644] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:23.111860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:23.111860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f759d-5afe-4f64-a041-61139357d38a service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:23.111860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ee4e6-9266-415d-af7d-d15278c584b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:23.111860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ee4e6-9266-415d-af7d-d15278c584b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:23.114987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ee4e6-9266-415d-af7d-d15278c584b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:23.114987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ee4e6-9266-415d-af7d-d15278c584b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:23.176050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ee4e6-9266-415d-af7d-d15278c584b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:23.178189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ee4e6-9266-415d-af7d-d15278c584b2 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:23.193734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 830/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:23.272067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b714b0a-ab81-4c66-8d36-57e4c33b4a2d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:23.284830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b714b0a-ab81-4c66-8d36-57e4c33b4a2d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:23.294981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:23.295429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:23.297431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:23.298064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:23.309435 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:23.317423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:23.320875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:23.350923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 816/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:23.921129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694424b-343b-4ab8-a6ce-845da562b601 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:23.921638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694424b-343b-4ab8-a6ce-845da562b601 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:23.922536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694424b-343b-4ab8-a6ce-845da562b601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:23.981888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-a694424b-343b-4ab8-a6ce-845da562b601 None None] MFA Rules not processed for user `e4dff416fce346b5b7679837bb7a5b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:23.984945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a694424b-343b-4ab8-a6ce-845da562b601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:24.174309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 831/1647] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:24.176573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:24.310595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed870c2-293d-4ba7-9484-1faeafa0b22b service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:24.316221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:24.317006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:24.317359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:24.317716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:24.318483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:24.332449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:24.351653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:24.584092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 817/1648] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:24 2022] POST /identity/v3/projects => generated 405 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:24.589717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694424b-343b-4ab8-a6ce-845da562b601 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:24.611622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694424b-343b-4ab8-a6ce-845da562b601 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:24.683352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:24.705554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:24.830499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:24.831503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:24.832117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:24.832598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:24.833510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:24.845916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:24.904426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675d08b-c07c-4765-bd7f-64cd420c2b55 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:24.936132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:24.937221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:24.938096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:24.938437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] PATH_INFO: `/v3/users/e2304cd0c41b49c580671cc359b1122d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:24.944361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2304cd0c41b49c580671cc359b1122d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:24.944664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: WARNING py.warnings [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:49:24.944664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: warnings.warn(msg) Jul 20 17:49:24.944664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: Jul 20 17:49:24.957518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:24.990381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 832/1649] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:24 2022] POST /identity/v3/users => generated 309 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:24.991477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:25.019922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655196-b382-4e77-8ab4-fa28f3507e85 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:25.040321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:25.041926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:25.042852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:25.044626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:25.045682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:25.049115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:25.086469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 833/1650] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:25.087872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:25.108468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cba53c-2158-479d-94b7-6419570cc0ba admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:25.113404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:25.114556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:25.114938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:25.115765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] PATH_INFO: `/v3/projects/0eaeee2da5be416cb52f3fe0a96ba2e4/users/96d8af187abc4f8e846a86b14929c900/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:25.283020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eaeee2da5be416cb52f3fe0a96ba2e4, user_id=96d8af187abc4f8e846a86b14929c900, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:25.292397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:25.376207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 834/1651] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:25 2022] PUT /identity/v3/projects/0eaeee2da5be416cb52f3fe0a96ba2e4/users/96d8af187abc4f8e846a86b14929c900/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:49:25.377637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:25.488671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2304cd0c41b49c580671cc359b1122d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:25.528990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67b1f1-99d2-4918-b05b-50906f3b15ba admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:25.536526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b7f14-c34c-40ca-9758-a2bb1fc6e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:25.537865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7f14-c34c-40ca-9758-a2bb1fc6e147 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:25.538259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7f14-c34c-40ca-9758-a2bb1fc6e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:25.538546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7f14-c34c-40ca-9758-a2bb1fc6e147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:25.539433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b7f14-c34c-40ca-9758-a2bb1fc6e147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:25.541131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b7f14-c34c-40ca-9758-a2bb1fc6e147 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:25.542025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2304cd0c41b49c580671cc359b1122d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:25.543257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2304cd0c41b49c580671cc359b1122d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:25.544756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2304cd0c41b49c580671cc359b1122d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:25.572071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287909e-a8fe-48da-9106-5a8e1a01e36a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2304cd0c41b49c580671cc359b1122d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:25.586148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 835/1652] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:25.588185 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29aa7e-0d4e-470f-8e84-236b4579e127 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:25.591040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29aa7e-0d4e-470f-8e84-236b4579e127 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:25.591435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29aa7e-0d4e-470f-8e84-236b4579e127 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:25.594327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 836/1653] 10.222.0.42 () {54 vars in 962 bytes} [Wed Jul 20 17:49:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:25.595138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29aa7e-0d4e-470f-8e84-236b4579e127 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:25.597314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0e29aa7e-0d4e-470f-8e84-236b4579e127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:25.619265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29aa7e-0d4e-470f-8e84-236b4579e127 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:25.635361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcf9b7-aa30-4d42-9a06-dfe64cc96b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:25.644156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf9b7-aa30-4d42-9a06-dfe64cc96b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:25.644481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf9b7-aa30-4d42-9a06-dfe64cc96b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:25.644783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcf9b7-aa30-4d42-9a06-dfe64cc96b0b admin admin] PATH_INFO: `/v3/projects/0eaeee2da5be416cb52f3fe0a96ba2e4/users/96d8af187abc4f8e846a86b14929c900/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:25.648758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcf9b7-aa30-4d42-9a06-dfe64cc96b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eaeee2da5be416cb52f3fe0a96ba2e4, user_id=96d8af187abc4f8e846a86b14929c900, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:25.651505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcf9b7-aa30-4d42-9a06-dfe64cc96b0b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:25.690077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 818/1654] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:24 2022] DELETE /identity/v3/users/e2304cd0c41b49c580671cc359b1122d => generated 0 bytes in 1008 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:25.693527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2db2-429f-43da-acca-2fbc6d6d1559 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:25.693960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2db2-429f-43da-acca-2fbc6d6d1559 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:25.694554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2db2-429f-43da-acca-2fbc6d6d1559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:25.705049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 837/1655] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:25 2022] PUT /identity/v3/projects/0eaeee2da5be416cb52f3fe0a96ba2e4/users/96d8af187abc4f8e846a86b14929c900/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:25.707067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:25.707362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:25.707642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:25.757106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c78e2db2-429f-43da-acca-2fbc6d6d1559 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:25.758608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c78e2db2-429f-43da-acca-2fbc6d6d1559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:25.761895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:25.764104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:25.940849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 838/1656] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:25.960083 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:25.961557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:26.003917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 819/1657] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Jul 20 17:49:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:26.005527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d5437-292d-428b-a829-616413c9a62f None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:26.005822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d5437-292d-428b-a829-616413c9a62f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:26.009048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d5437-292d-428b-a829-616413c9a62f None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:26.034118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:26.034211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 820/1658] 10.222.0.42 () {54 vars in 912 bytes} [Wed Jul 20 17:49:26 2022] GET /identity => generated 270 bytes in 29 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:26.035393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:26.035844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:26.036258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:26.076298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1eac3-3e42-4938-8687-91bdda20bed6 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:26.081858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:26.082384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:26.085422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:26.085979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:26.086391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:26.087002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:26.117907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 821/1659] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:49:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:26.135628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:26.136337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:26.137390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:26.137835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:26.155870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 839/1660] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:49:26.289082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84102f73-4837-4804-8d3b-0d8b7dc9f3c4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:26.302288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789b32d-83ef-4db7-9a96-1c2d80b841f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:26.303351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b32d-83ef-4db7-9a96-1c2d80b841f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:26.303893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b32d-83ef-4db7-9a96-1c2d80b841f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:26.306361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b32d-83ef-4db7-9a96-1c2d80b841f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:26.339285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789b32d-83ef-4db7-9a96-1c2d80b841f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:26.341111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789b32d-83ef-4db7-9a96-1c2d80b841f3 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:26.378206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 822/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:49:26.607729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:26.609695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:26.644038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926e8a-4dff-487b-aaba-afe3c43f8e26 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:26.658932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:26.659091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:26.660110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:26.660578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:26.661272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] PATH_INFO: `/v3/users/d1d9a99ed3e04d9782a4cfdd41b00376` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:26.663523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d9a99ed3e04d9782a4cfdd41b00376)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:26.670078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:26.999167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d9a99ed3e04d9782a4cfdd41b00376'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.002027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d9a99ed3e04d9782a4cfdd41b00376'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.002839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d9a99ed3e04d9782a4cfdd41b00376'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.021202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d9a99ed3e04d9782a4cfdd41b00376'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.156114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d9a99ed3e04d9782a4cfdd41b00376'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.205169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 840/1662] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:26 2022] DELETE /identity/v3/users/d1d9a99ed3e04d9782a4cfdd41b00376 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:27.209780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:27.210374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:27.210850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:27.256590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:27.258878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:27.462738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 823/1663] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:27.515809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:27.518287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:27.521589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:27.524405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:27.561400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3a36f-16b7-47ac-9ed5-a4a04b62b444 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:27.600926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:27.605064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:27.605417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:27.605746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:27.606727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] PATH_INFO: `/v3/projects/5fd29c713a91460b92d3926b0ec56200` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:27.610454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd29c713a91460b92d3926b0ec56200)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:27.616681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:27.710728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f8185-3f6b-496f-89dc-3fb623becae1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:27.717918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:27.719052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:27.719553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:27.720320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:27.742373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:27.744541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:27.771629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 841/1664] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:27.867631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd29c713a91460b92d3926b0ec56200', 'request_id': 'req-4c046407-82de-4928-b34e-ce0fecfe3c9b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.881041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd29c713a91460b92d3926b0ec56200', 'request_id': 'req-4c046407-82de-4928-b34e-ce0fecfe3c9b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.896200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd29c713a91460b92d3926b0ec56200', 'request_id': 'req-4c046407-82de-4928-b34e-ce0fecfe3c9b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.995066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] The token cache is being invalidate because project 5fd29c713a91460b92d3926b0ec56200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:27.995358 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:27.998108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:28.000285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 824/1665] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:27 2022] DELETE /identity/v3/projects/5fd29c713a91460b92d3926b0ec56200 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:28.004964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:28.033568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad747d18-0409-4035-ab2c-97f891671f15 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:28.196482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:28.202294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:28.202827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:28.204367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] PATH_INFO: `/v3/users/f9815c93377d4a4aaaf02e946083a6d9` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:28.206688 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9815c93377d4a4aaaf02e946083a6d9)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:28.215275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:28.504217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9815c93377d4a4aaaf02e946083a6d9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:28.505463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9815c93377d4a4aaaf02e946083a6d9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:28.508011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9815c93377d4a4aaaf02e946083a6d9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:28.525345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9815c93377d4a4aaaf02e946083a6d9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:28.616700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9815c93377d4a4aaaf02e946083a6d9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:28.651202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 842/1666] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:28 2022] DELETE /identity/v3/users/f9815c93377d4a4aaaf02e946083a6d9 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:29.617398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:29.618906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:29.626187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:29.628835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:29.648214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c046407-82de-4928-b34e-ce0fecfe3c9b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:29.806052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57c201-b912-424d-965e-67ebd572dbdd admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:29.817591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:29.818726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:29.821891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:29.822395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] PATH_INFO: `/v3/projects/bc61628523b44a37b6fc427205136ad9` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:29.825294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc61628523b44a37b6fc427205136ad9)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:29.826936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: WARNING py.warnings [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 17:49:29.826936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: warnings.warn(msg) Jul 20 17:49:29.826936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: Jul 20 17:49:29.830992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:29.852508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:29.853668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:29.854544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:29.854986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] PATH_INFO: `/v3/projects/7fddbb32c1f443348ed53bcd633ae412` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:29.857423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fddbb32c1f443348ed53bcd633ae412)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:29.859826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:30.067691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fddbb32c1f443348ed53bcd633ae412', 'request_id': 'req-9abc6de9-182c-4407-90c3-31d1e5bc5704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.084280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc61628523b44a37b6fc427205136ad9', 'request_id': 'req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.084677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fddbb32c1f443348ed53bcd633ae412', 'request_id': 'req-9abc6de9-182c-4407-90c3-31d1e5bc5704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.108204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fddbb32c1f443348ed53bcd633ae412', 'request_id': 'req-9abc6de9-182c-4407-90c3-31d1e5bc5704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.120984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc61628523b44a37b6fc427205136ad9', 'request_id': 'req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.139204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:30.206380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] The token cache is being invalidate because project 7fddbb32c1f443348ed53bcd633ae412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:30.206666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.208855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9abc6de9-182c-4407-90c3-31d1e5bc5704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.210719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 825/1667] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:29 2022] DELETE /identity/v3/projects/7fddbb32c1f443348ed53bcd633ae412 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:30.212322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:30.212824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:30.213269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:30.272618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] MFA Rules not processed for user `311c19d7062648f78ba4c2090c58f14d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:30.273867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:30.314682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc61628523b44a37b6fc427205136ad9', 'request_id': 'req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.347965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] The token cache is being invalidate because project bc61628523b44a37b6fc427205136ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:30.349431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.349759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-46c3f6fe-17d1-48c2-9c49-aa7e6153f8f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.351643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 843/1668] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:29 2022] DELETE /identity/v3/projects/bc61628523b44a37b6fc427205136ad9 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:30.353146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:30.354350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:30.354924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:30.398613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 826/1669] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:30.399873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:30.400202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:30.401624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:30.408929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:30.479390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b4ced-2a60-45a7-883f-c3cf78af8d76 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:30.484301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:30.485906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:30.486524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:30.486915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] PATH_INFO: `/v3/users/e0d35514496747769a822aa9f1157127` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:30.488411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d35514496747769a822aa9f1157127)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:30.490900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:30.545028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 844/1670] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:30.546149 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:30.547715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:30.553826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:30.671338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b7c61-2011-4c2e-9fd1-a9eba521d086 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:30.683954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0162ef-54c0-4584-90b7-631860b74c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:30.685157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0162ef-54c0-4584-90b7-631860b74c3a service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:30.685763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0162ef-54c0-4584-90b7-631860b74c3a service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:30.686234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0162ef-54c0-4584-90b7-631860b74c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:30.697709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0162ef-54c0-4584-90b7-631860b74c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:30.699886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0162ef-54c0-4584-90b7-631860b74c3a service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:30.721061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d35514496747769a822aa9f1157127'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.732261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 845/1671] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:30.735037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07411e-589c-4861-9cf4-9e6f3dfae1a5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:30.735490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07411e-589c-4861-9cf4-9e6f3dfae1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:30.736721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07411e-589c-4861-9cf4-9e6f3dfae1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:30.746237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d35514496747769a822aa9f1157127'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.747928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d35514496747769a822aa9f1157127'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.748768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d35514496747769a822aa9f1157127'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.765242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d35514496747769a822aa9f1157127'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:30.793075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-fa07411e-589c-4861-9cf4-9e6f3dfae1a5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:30.795089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fa07411e-589c-4861-9cf4-9e6f3dfae1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:30.822201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 846/1672] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:30.825648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07411e-589c-4861-9cf4-9e6f3dfae1a5 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:30.849325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07411e-589c-4861-9cf4-9e6f3dfae1a5 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:30.860139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 827/1673] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:30 2022] DELETE /identity/v3/users/e0d35514496747769a822aa9f1157127 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:30.865104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a3c4c-e25c-43f4-9358-30226234534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:30.867979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a3c4c-e25c-43f4-9358-30226234534c admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:30.868275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a3c4c-e25c-43f4-9358-30226234534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:30.870701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a3c4c-e25c-43f4-9358-30226234534c admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:30.880636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a3c4c-e25c-43f4-9358-30226234534c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:30.881044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a3c4c-e25c-43f4-9358-30226234534c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:30.892092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a3c4c-e25c-43f4-9358-30226234534c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:30.923266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 847/1674] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:30.929919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:30.994474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31165a-1ed1-4fba-940a-592e79066e47 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:30.994859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31165a-1ed1-4fba-940a-592e79066e47 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:30.995286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31165a-1ed1-4fba-940a-592e79066e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:31.073865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4a31165a-1ed1-4fba-940a-592e79066e47 None None] MFA Rules not processed for user `96d8af187abc4f8e846a86b14929c900`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:31.073865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4a31165a-1ed1-4fba-940a-592e79066e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:31.105621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c733-060d-4f57-a42c-fdc3106c16f5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:31.113923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:31.114764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:31.115086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:31.115378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:31.116093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:31.118067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:31.230779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 828/1675] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:30 2022] POST /identity/v3/projects => generated 387 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:31.235947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:31.269194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 848/1676] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:31.361933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f274a5c-0b98-44f1-b345-f13ea8e0e8ee admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:31.378791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:31.379521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:31.379786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:31.381747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:31.382589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:31.385027 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:31.553505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 829/1677] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:31 2022] POST /identity/v3/users => generated 294 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:31.554920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31165a-1ed1-4fba-940a-592e79066e47 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:31.576889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31165a-1ed1-4fba-940a-592e79066e47 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:31.595040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:31.595839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:31.596602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:31.598002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:31.598243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:31.601289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:31.623077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 849/1678] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:31.626207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:31.650836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c708d86-96fe-40e8-a11d-fc788575f9c9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:31.670085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02af218-5f3b-4862-9cfb-7e8a0c8c1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:31.670085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02af218-5f3b-4862-9cfb-7e8a0c8c1938 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:31.670085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02af218-5f3b-4862-9cfb-7e8a0c8c1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:31.670085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02af218-5f3b-4862-9cfb-7e8a0c8c1938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:31.670541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02af218-5f3b-4862-9cfb-7e8a0c8c1938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:31.670541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02af218-5f3b-4862-9cfb-7e8a0c8c1938 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:31.757889 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 830/1679] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:31.764133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:31.782822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561f90b-8517-4e3b-94cd-e7cdf6466190 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:31.801655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd03fb7-d2be-41be-b14b-3d8233d3c7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:31.803085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd03fb7-d2be-41be-b14b-3d8233d3c7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:31.803312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd03fb7-d2be-41be-b14b-3d8233d3c7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:31.803695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd03fb7-d2be-41be-b14b-3d8233d3c7d0 admin admin] PATH_INFO: `/v3/projects/29215561b45a4fc78c7538134857b275/users/8c22854b3b234d879f233e040c8662b2/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:31.846062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd03fb7-d2be-41be-b14b-3d8233d3c7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29215561b45a4fc78c7538134857b275, user_id=8c22854b3b234d879f233e040c8662b2, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:31.851037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd03fb7-d2be-41be-b14b-3d8233d3c7d0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:32.059740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 850/1680] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:31 2022] PUT /identity/v3/projects/29215561b45a4fc78c7538134857b275/users/8c22854b3b234d879f233e040c8662b2/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:32.084583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:32.084583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:32.084583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:32.143855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] MFA Rules not processed for user `8c22854b3b234d879f233e040c8662b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:32.153732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:32.193949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa39a2-a57f-4b58-bdf0-2783c6f7e8da None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:32.200823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa39a2-a57f-4b58-bdf0-2783c6f7e8da None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:32.200823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa39a2-a57f-4b58-bdf0-2783c6f7e8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:32.295768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4caa39a2-a57f-4b58-bdf0-2783c6f7e8da None None] MFA Rules not processed for user `3077a7934f9c4407a70a8790b8da9cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:32.299381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4caa39a2-a57f-4b58-bdf0-2783c6f7e8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:32.391096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 831/1681] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:32.393629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:32.411426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:32.490072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 851/1682] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:49:32.490072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa39a2-a57f-4b58-bdf0-2783c6f7e8da None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:32.543276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9b22c-ddd9-4d93-a50d-a6befad9c228 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:32.551708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:32.554103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:32.555629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:32.556031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] PATH_INFO: `/v3/projects/467110c550b244de9e996ce21319c333` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:32.559427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=467110c550b244de9e996ce21319c333)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:32.563180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:32.631842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa39a2-a57f-4b58-bdf0-2783c6f7e8da None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:32.637916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:32.638745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:32.639136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:32.639618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:32.641155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:32.643326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:32.771120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467110c550b244de9e996ce21319c333', 'request_id': 'req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:32.778824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467110c550b244de9e996ce21319c333', 'request_id': 'req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:32.791205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467110c550b244de9e996ce21319c333', 'request_id': 'req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:32.817829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 852/1683] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:32 2022] POST /identity/v3/projects => generated 385 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:32.820109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:32.827267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:32.954551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] The token cache is being invalidate because project 467110c550b244de9e996ce21319c333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:32.955783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:32.957627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:32.960192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 832/1684] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:32 2022] DELETE /identity/v3/projects/467110c550b244de9e996ce21319c333 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:32.961746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:32.967375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.044824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff52c0-29e3-4b73-8a84-05311bb2e811 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:33.070383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c3fc-ed78-440d-a51b-d21d3cdd2ae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:33.070383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c3fc-ed78-440d-a51b-d21d3cdd2ae7 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.070383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c3fc-ed78-440d-a51b-d21d3cdd2ae7 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.070383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c3fc-ed78-440d-a51b-d21d3cdd2ae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.070914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b356c3fc-ed78-440d-a51b-d21d3cdd2ae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.106256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb286a65-ee2e-4d27-80f8-9e83f5f1bbd2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:33.217245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356c3fc-ed78-440d-a51b-d21d3cdd2ae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:33.219437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356c3fc-ed78-440d-a51b-d21d3cdd2ae7 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:33.234580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:33.258269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 853/1685] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:33.259926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.260220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.260525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.296399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:33.297952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.299527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117d077-cb77-49aa-a27c-19948aec253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:33.300919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117d077-cb77-49aa-a27c-19948aec253f admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.301200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117d077-cb77-49aa-a27c-19948aec253f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.301476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117d077-cb77-49aa-a27c-19948aec253f admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.307920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117d077-cb77-49aa-a27c-19948aec253f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:33.321306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117d077-cb77-49aa-a27c-19948aec253f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:33.325170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 854/1686] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:33.326077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:33.329232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.365096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adbafcf-b682-4f5f-b7eb-5459fe6483a4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:33.379300 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:33.380265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.380589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.381160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] PATH_INFO: `/v3/users/4b000575206c4b98889a8a26c98c5d3a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.384251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b000575206c4b98889a8a26c98c5d3a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:33.386432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:33.453107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 833/1687] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:32 2022] POST /identity/v3/users => generated 293 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:33.457683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfb77e-ec39-47a5-92c7-524a33bf9803 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.457982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfb77e-ec39-47a5-92c7-524a33bf9803 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.458281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfb77e-ec39-47a5-92c7-524a33bf9803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.500099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-24cfb77e-ec39-47a5-92c7-524a33bf9803 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:33.508206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-24cfb77e-ec39-47a5-92c7-524a33bf9803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.522036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 834/1688] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:33.523788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0e28-bc76-4558-a1be-b9ca7fdcc929 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.530183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0e28-bc76-4558-a1be-b9ca7fdcc929 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.530183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0e28-bc76-4558-a1be-b9ca7fdcc929 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.530183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 835/1689] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:33.530183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0e28-bc76-4558-a1be-b9ca7fdcc929 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:33.563633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0e28-bc76-4558-a1be-b9ca7fdcc929 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:33.578767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:33.579461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.580909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.593354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.593354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:33.593354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:33.614833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 836/1690] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:33.614833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:33.619187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:33.641840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3398f4a-c986-4546-bdae-9b0b252433d8 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:33.666556 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b000575206c4b98889a8a26c98c5d3a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:33.671454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b000575206c4b98889a8a26c98c5d3a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:33.674407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b000575206c4b98889a8a26c98c5d3a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:33.674563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52b2e9-b946-43a1-9496-8886ddac4854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:33.676530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52b2e9-b946-43a1-9496-8886ddac4854 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.676920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52b2e9-b946-43a1-9496-8886ddac4854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.677336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52b2e9-b946-43a1-9496-8886ddac4854 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.680996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52b2e9-b946-43a1-9496-8886ddac4854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:33.681454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52b2e9-b946-43a1-9496-8886ddac4854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:33.690459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52b2e9-b946-43a1-9496-8886ddac4854 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:33.696823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b000575206c4b98889a8a26c98c5d3a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:33.728504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 837/1691] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:33.729049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:33.729349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:33.729626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:33.789912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:33.791148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.841024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b000575206c4b98889a8a26c98c5d3a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:33.877840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 855/1692] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:33 2022] DELETE /identity/v3/users/4b000575206c4b98889a8a26c98c5d3a => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:33.885756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:33.885756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.886074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 838/1693] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:33.886074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:33.886693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:33.946791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dc013-16e2-45ec-9e9c-b0699b350742 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:34.041270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae15c5-ff71-47d0-acfa-6a51f8c6dd62 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:34.047727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0642688-5a88-4bf5-93bc-b435991c74e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:34.048274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76224e-523a-46e8-9618-7d9d28d2b9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:34.048511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0642688-5a88-4bf5-93bc-b435991c74e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.048955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0642688-5a88-4bf5-93bc-b435991c74e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.049402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0642688-5a88-4bf5-93bc-b435991c74e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.050254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0642688-5a88-4bf5-93bc-b435991c74e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:34.051813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76224e-523a-46e8-9618-7d9d28d2b9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.052106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76224e-523a-46e8-9618-7d9d28d2b9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.052390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76224e-523a-46e8-9618-7d9d28d2b9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.053368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76224e-523a-46e8-9618-7d9d28d2b9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:34.053630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0642688-5a88-4bf5-93bc-b435991c74e6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:34.055485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76224e-523a-46e8-9618-7d9d28d2b9e6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:34.080984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 856/1694] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:34.085992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e5cb1-93fe-4b0e-bde8-4faf52009240 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.086553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e5cb1-93fe-4b0e-bde8-4faf52009240 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.100184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e5cb1-93fe-4b0e-bde8-4faf52009240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.146671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 839/1695] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:33 2022] POST /identity/v3/projects => generated 377 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:34.151950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f508541-8fe2-4fa8-9474-ac895c67f55a None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.152246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f508541-8fe2-4fa8-9474-ac895c67f55a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.152562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f508541-8fe2-4fa8-9474-ac895c67f55a None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.161368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 840/1696] 10.222.0.42 () {54 vars in 961 bytes} [Wed Jul 20 17:49:34 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:34.164888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73b07-26d0-434f-a91c-555bd4ec5e47 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.165186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73b07-26d0-434f-a91c-555bd4ec5e47 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.165467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73b07-26d0-434f-a91c-555bd4ec5e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.197170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-589e5cb1-93fe-4b0e-bde8-4faf52009240 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:34.198932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-589e5cb1-93fe-4b0e-bde8-4faf52009240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:34.246526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-51d73b07-26d0-434f-a91c-555bd4ec5e47 None None] MFA Rules not processed for user `137e21587db048529f771d750da5b208`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:34.250412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-51d73b07-26d0-434f-a91c-555bd4ec5e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:34.295767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 857/1697] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:34.297247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e5cb1-93fe-4b0e-bde8-4faf52009240 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:34.311307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e5cb1-93fe-4b0e-bde8-4faf52009240 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:34.317497 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e224d-8e68-4a9f-bb00-324da5437591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:34.318430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e224d-8e68-4a9f-bb00-324da5437591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.318717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e224d-8e68-4a9f-bb00-324da5437591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.318999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e224d-8e68-4a9f-bb00-324da5437591 admin admin] PATH_INFO: `/v3/projects/308aaff4a31e4caa892914e2adca45ac/users/45529c5ab4c347e4b0736d7691467ed8/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.343485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e224d-8e68-4a9f-bb00-324da5437591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=308aaff4a31e4caa892914e2adca45ac, user_id=45529c5ab4c347e4b0736d7691467ed8, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:34.349539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e224d-8e68-4a9f-bb00-324da5437591 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:34.443214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 841/1698] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:34.444849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73b07-26d0-434f-a91c-555bd4ec5e47 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:34.469242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73b07-26d0-434f-a91c-555bd4ec5e47 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:34.492418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 858/1699] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:34 2022] PUT /identity/v3/projects/308aaff4a31e4caa892914e2adca45ac/users/45529c5ab4c347e4b0736d7691467ed8/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:34.494590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.495104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.495445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.524520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:34.525690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:34.674131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8124f-a286-44d6-b7ac-f8047b59bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:34.676386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8124f-a286-44d6-b7ac-f8047b59bb7e admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.676718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8124f-a286-44d6-b7ac-f8047b59bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.679213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8124f-a286-44d6-b7ac-f8047b59bb7e admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.679997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8124f-a286-44d6-b7ac-f8047b59bb7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:34.691768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8124f-a286-44d6-b7ac-f8047b59bb7e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:34.706646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 859/1700] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Jul 20 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:34.707549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:34.710592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:34.814484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 842/1701] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:34 2022] POST /identity/v3/users => generated 289 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:34.817770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c16f45-b841-489d-a07f-d020852e6818 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.819557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c16f45-b841-489d-a07f-d020852e6818 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.821101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c16f45-b841-489d-a07f-d020852e6818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.827433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e742736-3991-4414-a042-7ce3ca8760c2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:34.843163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cb35d-cbac-4918-a3c6-2223fc939259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:34.845172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb35d-cbac-4918-a3c6-2223fc939259 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.845638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb35d-cbac-4918-a3c6-2223fc939259 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.846080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cb35d-cbac-4918-a3c6-2223fc939259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.866222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cb35d-cbac-4918-a3c6-2223fc939259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:34.866896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741cb35d-cbac-4918-a3c6-2223fc939259 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:34.879657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-86c16f45-b841-489d-a07f-d020852e6818 None None] MFA Rules not processed for user `137e21587db048529f771d750da5b208`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:34.882836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-86c16f45-b841-489d-a07f-d020852e6818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:34.905338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 860/1702] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:34.909498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b892276-fe32-4363-ba20-ef599926191c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.910005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b892276-fe32-4363-ba20-ef599926191c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.910446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b892276-fe32-4363-ba20-ef599926191c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:34.956912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 843/1703] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:34.958076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c16f45-b841-489d-a07f-d020852e6818 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:34.973697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c16f45-b841-489d-a07f-d020852e6818 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:34.982192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:34.985597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:34.985597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:34.985597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.007599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.010273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.014272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-9b892276-fe32-4363-ba20-ef599926191c None None] MFA Rules not processed for user `45529c5ab4c347e4b0736d7691467ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:35.014272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9b892276-fe32-4363-ba20-ef599926191c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:35.024572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 844/1704] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:35.025850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:35.039695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544ace6-d6ee-4676-b4a9-c421a70eceab service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:35.045687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:35.047854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.048307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.048796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.049960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.061013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.115032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 845/1705] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:35.116023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:35.123383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:35.195414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 861/1706] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:35.196057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b892276-fe32-4363-ba20-ef599926191c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:35.220686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b892276-fe32-4363-ba20-ef599926191c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:35.231559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:35.232257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.233352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.233523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.234828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.237952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.254632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 862/1707] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:35.259020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:35.279361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb82e4-6d1d-46e0-bd8a-48d5df1f9c5e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:35.291693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b88b9-e8df-4896-8b86-b0bfc5de10a5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:35.303622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:35.305263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.305585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.305872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.306626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.311704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:35.312451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.312745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.313021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.317105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.343286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.349028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.365824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:35.390998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 846/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:35.392479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:35.428473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89f9dd-fdbc-4431-b4ca-95045fc6a64d service nova] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:35.449070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:35.449990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.450273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.450551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] PATH_INFO: `/v3/projects/f768f28d1cdc41d1afbd15edcdf14268/users/e829024f3bba48119b610328a92034a9/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.514816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 863/1709] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:35 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:35.516777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:35.523752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:35.523965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f768f28d1cdc41d1afbd15edcdf14268, user_id=e829024f3bba48119b610328a92034a9, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.526664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.670135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 847/1710] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:35 2022] PUT /identity/v3/projects/f768f28d1cdc41d1afbd15edcdf14268/users/e829024f3bba48119b610328a92034a9/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:35.671384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:35.693753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188771-caf6-4f49-8000-5acfca46aff3 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:35.799104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb41121-393f-47a5-b318-d7a1a25b9b41 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:35.810482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44c40-bae5-40d6-9907-a2c140aedc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:35.811753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44c40-bae5-40d6-9907-a2c140aedc3c admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.812827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44c40-bae5-40d6-9907-a2c140aedc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.813347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44c40-bae5-40d6-9907-a2c140aedc3c admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.814305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a44c40-bae5-40d6-9907-a2c140aedc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.818285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a44c40-bae5-40d6-9907-a2c140aedc3c admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.825413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:35.826702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.827200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.828620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] PATH_INFO: `/v3/users/5c5dba86648f44f3ad71ae095cad4afc` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:35.833241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5dba86648f44f3ad71ae095cad4afc)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:35.837461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:35.862389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:35.961971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 848/1711] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:35 2022] POST /identity/v3/users => generated 299 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:35.966492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:35.966492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:35.966492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:36.004849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] MFA Rules not processed for user `e829024f3bba48119b610328a92034a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:36.008829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:36.155214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5dba86648f44f3ad71ae095cad4afc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.159807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5dba86648f44f3ad71ae095cad4afc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.160373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5dba86648f44f3ad71ae095cad4afc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.185738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5dba86648f44f3ad71ae095cad4afc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.238012 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 849/1712] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:36.244422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:36.246464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:36.284039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270511af-dd53-4846-9d99-ca0e01889cdf None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:36.290607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:36.292089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:36.292629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:36.293846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] PATH_INFO: `/v3/users/e4dff416fce346b5b7679837bb7a5b5c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:36.295420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4dff416fce346b5b7679837bb7a5b5c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:36.298030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:36.309112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5dba86648f44f3ad71ae095cad4afc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.353990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 864/1713] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:35 2022] DELETE /identity/v3/users/5c5dba86648f44f3ad71ae095cad4afc => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:36.359853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:36.362826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:36.497947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602248e-0ee3-404c-86eb-7fca77e5f1d8 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:36.497947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:36.497947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:36.497947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:36.498479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:36.498479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:36.498479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:36.532840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 865/1714] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:36.535585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:36.538037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:36.570717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa8a0e-bfcd-4eb2-ae65-a3b918761ceb admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:36.585158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffea79-684e-48ab-af2c-850170ba914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:36.585866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffea79-684e-48ab-af2c-850170ba914f admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:36.586465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffea79-684e-48ab-af2c-850170ba914f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:36.586465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffea79-684e-48ab-af2c-850170ba914f admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:36.587390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffea79-684e-48ab-af2c-850170ba914f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:36.589772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffea79-684e-48ab-af2c-850170ba914f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:36.632913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4dff416fce346b5b7679837bb7a5b5c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.655003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4dff416fce346b5b7679837bb7a5b5c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.655825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4dff416fce346b5b7679837bb7a5b5c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.656632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4dff416fce346b5b7679837bb7a5b5c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.666528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b083fb06-b080-4745-9725-07a6acf8f414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4dff416fce346b5b7679837bb7a5b5c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:36.717451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 866/1715] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:36 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:36.721606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:36.722033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:36.722570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:36.758688 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:36.760912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:36.814279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 850/1716] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:36 2022] DELETE /identity/v3/users/e4dff416fce346b5b7679837bb7a5b5c => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:36.819694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15443a5c-ada0-42f3-b810-55710dfa9824 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:36.819694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15443a5c-ada0-42f3-b810-55710dfa9824 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:36.819694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15443a5c-ada0-42f3-b810-55710dfa9824 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:36.824947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 851/1717] 10.222.0.42 () {54 vars in 912 bytes} [Wed Jul 20 17:49:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:36.825955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15443a5c-ada0-42f3-b810-55710dfa9824 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:36.842654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-15443a5c-ada0-42f3-b810-55710dfa9824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:36.985731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 867/1718] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:36.985731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:36.992086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:37.035055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d396c-2361-4e0f-9071-aa89f417bc93 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:37.115795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15443a5c-ada0-42f3-b810-55710dfa9824 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:37.124242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092087b1-2e65-4360-abaf-8025b15ecfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:37.125285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092087b1-2e65-4360-abaf-8025b15ecfcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.125766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092087b1-2e65-4360-abaf-8025b15ecfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.129127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092087b1-2e65-4360-abaf-8025b15ecfcb admin admin] PATH_INFO: `/v3/projects/7a61cfabfa0c43eea5fd0f00636de434/users/53ea326186a1438d939a6555c7525175/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.182980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092087b1-2e65-4360-abaf-8025b15ecfcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a61cfabfa0c43eea5fd0f00636de434, user_id=53ea326186a1438d939a6555c7525175, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:37.193127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092087b1-2e65-4360-abaf-8025b15ecfcb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:37.219418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:37.220921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.221214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.221546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.222283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:37.225056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:37.301397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 852/1719] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:36 2022] PUT /identity/v3/projects/7a61cfabfa0c43eea5fd0f00636de434/users/53ea326186a1438d939a6555c7525175/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:37.305084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.305606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.306579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.325100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 868/1720] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:36 2022] POST /identity/v3/users => generated 294 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:37.327824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:37.333465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:37.354305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7012c8-c593-47e0-a796-b4b5effcd701 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:37.383608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] MFA Rules not processed for user `d1d18a38f45e4501914e22689f145fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:37.385194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:37.514931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:37.517644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.519307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.519712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] PATH_INFO: `/v3/projects/8eebf9df517849fcbc5a104d28c60257` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.525903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eebf9df517849fcbc5a104d28c60257)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:37.528313 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:37.533196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:37.565138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 853/1721] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:49:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:37.570043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:37.578946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:37.613373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c018c-b21f-47f3-9080-7b4a837db7c6 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:37.629197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:37.630042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.630409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.630757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.646925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:37.650961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:37.688970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 854/1722] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:37.690315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:37.715123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eebf9df517849fcbc5a104d28c60257', 'request_id': 'req-d2e0d9ce-c597-4950-872d-7fe430d282ef'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:37.716782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912e33-0bb4-482c-8466-188ac732c326 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:37.724614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eebf9df517849fcbc5a104d28c60257', 'request_id': 'req-d2e0d9ce-c597-4950-872d-7fe430d282ef'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:37.831782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:37.833493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.833959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.834415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.835349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:37.837361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:37.846246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eebf9df517849fcbc5a104d28c60257', 'request_id': 'req-d2e0d9ce-c597-4950-872d-7fe430d282ef'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:37.851164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 855/1723] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:37.852221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:37.865790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] The token cache is being invalidate because project 8eebf9df517849fcbc5a104d28c60257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:37.868194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:37.870581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:37.881348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 869/1724] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:37 2022] DELETE /identity/v3/projects/8eebf9df517849fcbc5a104d28c60257 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:37.881348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:37.886262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:37.890636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0267-7914-492e-908c-3a5fc705bf72 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:37.903118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:37.904568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.907639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.908240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.909197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:37.912161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:37.941157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 856/1725] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:37.943330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:37.951721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:37.978768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1b592-9d8b-4676-8f37-4e7667df4f3d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:37.992269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:37.996423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:37.997036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:37.997465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] PATH_INFO: `/v3/projects/073215a6db1a496385d71ae843886dd4` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:37.999210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=073215a6db1a496385d71ae843886dd4)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:38.004340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:38.052264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0d9ce-c597-4950-872d-7fe430d282ef admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:38.055969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:38.076239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:38.076239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:38.076239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:38.076239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:38.084800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:38.129870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:38.135361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:38.195431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '073215a6db1a496385d71ae843886dd4', 'request_id': 'req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.196018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 870/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:38.199382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:38.201240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:38.229661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '073215a6db1a496385d71ae843886dd4', 'request_id': 'req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.259460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '073215a6db1a496385d71ae843886dd4', 'request_id': 'req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.259637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba97c36-e1f6-4c5f-b5b1-662ed1930714 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:38.367986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:38.369418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:38.369857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:38.370292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] PATH_INFO: `/v3/projects/7a61cfabfa0c43eea5fd0f00636de434/users/53ea326186a1438d939a6555c7525175/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:38.379360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] The token cache is being invalidate because project 073215a6db1a496385d71ae843886dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:38.381996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.381996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.383399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 857/1727] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:37 2022] DELETE /identity/v3/projects/073215a6db1a496385d71ae843886dd4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:38.387934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:38.389769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a61cfabfa0c43eea5fd0f00636de434, user_id=53ea326186a1438d939a6555c7525175, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:38.392295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:38.405210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:38.422139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 871/1728] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:38 2022] PUT /identity/v3/projects/7a61cfabfa0c43eea5fd0f00636de434/users/53ea326186a1438d939a6555c7525175/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:38.423260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:38.427963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34efe-63b4-4fe6-a846-67a0a0dc1835 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:38.435843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:38.537672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:38.539288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:38.539934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:38.540475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] PATH_INFO: `/v3/users/bde3b4f7608c41b6b1f0b7f35541c007` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:38.543554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde3b4f7608c41b6b1f0b7f35541c007)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:38.548013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:38.571723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eb80b-8ad1-4878-bb9b-6db4b60ce8c5 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:38.573872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:38.580274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d18128-e34f-49d1-ad67-4107c3261300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:38.581389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d18128-e34f-49d1-ad67-4107c3261300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:38.582585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d18128-e34f-49d1-ad67-4107c3261300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:38.582925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d18128-e34f-49d1-ad67-4107c3261300 admin admin] PATH_INFO: `/v3/projects/ff857e68441944dda1e34365118a7fb9/users/91a2584742c649cbae59b6684e631e77/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:38.612175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d18128-e34f-49d1-ad67-4107c3261300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff857e68441944dda1e34365118a7fb9, user_id=91a2584742c649cbae59b6684e631e77, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:38.613623 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d18128-e34f-49d1-ad67-4107c3261300 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:38.749405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 872/1729] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:38 2022] PUT /identity/v3/projects/ff857e68441944dda1e34365118a7fb9/users/91a2584742c649cbae59b6684e631e77/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:38.751040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:38.751950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:38.751950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:38.769210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] MFA Rules not processed for user `311c19d7062648f78ba4c2090c58f14d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:38.770543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:38.801883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde3b4f7608c41b6b1f0b7f35541c007'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.848308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde3b4f7608c41b6b1f0b7f35541c007'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.849256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde3b4f7608c41b6b1f0b7f35541c007'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.850532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde3b4f7608c41b6b1f0b7f35541c007'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.870836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-701fd2a4-1100-468e-adc6-6dde9daacfda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde3b4f7608c41b6b1f0b7f35541c007'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:38.899964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 873/1730] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:38.899964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:38.899964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:38.962486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 858/1731] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:38 2022] DELETE /identity/v3/users/bde3b4f7608c41b6b1f0b7f35541c007 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:38.964033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:38.964569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:38.966152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:39.024230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:39.066423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] MFA Rules not processed for user `53ea326186a1438d939a6555c7525175`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:39.068952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:39.092216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8c8b5-ae2b-4abb-8e76-cbbcc100cb63 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:39.300729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:39.302474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:39.303234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:39.304576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] PATH_INFO: `/v3/users/96d8af187abc4f8e846a86b14929c900` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:39.306371 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d8af187abc4f8e846a86b14929c900)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:39.317922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:39.372328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 859/1732] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:39.377019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:39.381632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:39.401785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8f631-968f-492d-ba35-99aa37fd3e4b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:39.407719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:39.412975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:39.412975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:39.412975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:39.412975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:39.417615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:39.432909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 860/1733] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:39.434170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:39.444873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:39.566343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bb684-1699-4a3d-813a-94b0e96e4b1f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:39.576942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:39.577625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:39.584264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:39.584597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:39.587102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d8af187abc4f8e846a86b14929c900'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:39.588617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d8af187abc4f8e846a86b14929c900'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:39.589466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d8af187abc4f8e846a86b14929c900'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:39.595236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:39.610127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d8af187abc4f8e846a86b14929c900'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:39.624417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:39.627456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:39.655162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 861/1734] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:39.656161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:39.660250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:39.721342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d8af187abc4f8e846a86b14929c900'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:39.794356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 874/1735] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:38 2022] DELETE /identity/v3/users/96d8af187abc4f8e846a86b14929c900 => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:39.799212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:39.805074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:39.808960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72aae73-28c2-4f03-9fd1-f26b559e44fa service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:39.928844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:39.929549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:39.929842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:39.930119 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:39.938028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:39.995975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d6c8f-790b-4d36-855e-4076ce3da5ca admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:40.006563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67ed03-219a-48b3-a905-1f8a89089138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:40.008230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67ed03-219a-48b3-a905-1f8a89089138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:40.008559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67ed03-219a-48b3-a905-1f8a89089138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:40.008849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67ed03-219a-48b3-a905-1f8a89089138 admin admin] PATH_INFO: `/v3/projects/ff857e68441944dda1e34365118a7fb9/users/91a2584742c649cbae59b6684e631e77/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:40.012684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67ed03-219a-48b3-a905-1f8a89089138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff857e68441944dda1e34365118a7fb9, user_id=91a2584742c649cbae59b6684e631e77, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:40.021423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67ed03-219a-48b3-a905-1f8a89089138 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:40.057663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 875/1736] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:39 2022] PUT /identity/v3/projects/ff857e68441944dda1e34365118a7fb9/users/91a2584742c649cbae59b6684e631e77/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:40.067133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171982-b98b-4728-b6b6-5c2179ff69bc None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:40.067351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171982-b98b-4728-b6b6-5c2179ff69bc None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:40.076512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171982-b98b-4728-b6b6-5c2179ff69bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:40.110546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b1171982-b98b-4728-b6b6-5c2179ff69bc None None] MFA Rules not processed for user `3077a7934f9c4407a70a8790b8da9cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:40.112143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b1171982-b98b-4728-b6b6-5c2179ff69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:40.129951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:40.131717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:40.159750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 862/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:40.160991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:40.165225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:40.175922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:40.202205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07069b-2e3f-491d-8125-466738f7e509 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:40.259321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 876/1738] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:40.262826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:40.263276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:40.264555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:40.370993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] MFA Rules not processed for user `91a2584742c649cbae59b6684e631e77`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:40.372216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:40.372359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:40.378289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:40.378289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:40.378289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] PATH_INFO: `/v3/projects/2731e64fe89e484fa08dc52786fe4dd5` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:40.378289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2731e64fe89e484fa08dc52786fe4dd5)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:40.392552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:40.575573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 877/1739] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:40.590396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:40.595800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:40.619519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:40.667180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2731e64fe89e484fa08dc52786fe4dd5', 'request_id': 'req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:40.688402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2731e64fe89e484fa08dc52786fe4dd5', 'request_id': 'req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:40.695254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd4555-bd5e-4d34-a6f3-0c842c94b2ef None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:40.713222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2731e64fe89e484fa08dc52786fe4dd5', 'request_id': 'req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:40.881475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] The token cache is being invalidate because project 2731e64fe89e484fa08dc52786fe4dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:40.882783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:40.882923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:40.885075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 863/1740] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:40 2022] DELETE /identity/v3/projects/2731e64fe89e484fa08dc52786fe4dd5 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:40.892217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:40.900459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:40.914349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:40.915461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:40.916084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:40.916552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] PATH_INFO: `/v3/projects/c3d0a9d63f57472fa588adf60a13ef3a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:40.918510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d0a9d63f57472fa588adf60a13ef3a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:40.921475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:40.959520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4d3d4-414f-400b-9a1a-5b1a37958f64 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:40.975309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:40.976668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:40.980103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:40.980688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] PATH_INFO: `/v3/projects/0eaeee2da5be416cb52f3fe0a96ba2e4` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:40.985302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eaeee2da5be416cb52f3fe0a96ba2e4)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:40.987450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.161153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d0a9d63f57472fa588adf60a13ef3a', 'request_id': 'req-34451305-f161-4ea6-86d6-8b29a4a666ec'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.169815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d0a9d63f57472fa588adf60a13ef3a', 'request_id': 'req-34451305-f161-4ea6-86d6-8b29a4a666ec'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.233126 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eaeee2da5be416cb52f3fe0a96ba2e4', 'request_id': 'req-aabe40c7-3647-41e6-9fa3-5c13b25e7582'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.244751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eaeee2da5be416cb52f3fe0a96ba2e4', 'request_id': 'req-aabe40c7-3647-41e6-9fa3-5c13b25e7582'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.254643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eaeee2da5be416cb52f3fe0a96ba2e4', 'request_id': 'req-aabe40c7-3647-41e6-9fa3-5c13b25e7582'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.306937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d0a9d63f57472fa588adf60a13ef3a', 'request_id': 'req-34451305-f161-4ea6-86d6-8b29a4a666ec'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.318926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] The token cache is being invalidate because project c3d0a9d63f57472fa588adf60a13ef3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:41.319217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.319870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.321496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 878/1741] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:40 2022] DELETE /identity/v3/projects/c3d0a9d63f57472fa588adf60a13ef3a => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:41.322309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.323731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.419380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34451305-f161-4ea6-86d6-8b29a4a666ec admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:41.423418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] The token cache is being invalidate because project 0eaeee2da5be416cb52f3fe0a96ba2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:41.423931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.424950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aabe40c7-3647-41e6-9fa3-5c13b25e7582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:41.426917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 864/1742] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:40 2022] DELETE /identity/v3/projects/0eaeee2da5be416cb52f3fe0a96ba2e4 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:41.428636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3085c-838e-48f2-bbcf-b6c66a8077ce None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.429106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3085c-838e-48f2-bbcf-b6c66a8077ce None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.429563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3085c-838e-48f2-bbcf-b6c66a8077ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.461469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-21f3085c-838e-48f2-bbcf-b6c66a8077ce None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:41.462767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-21f3085c-838e-48f2-bbcf-b6c66a8077ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.493458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359ddfe-ccb7-4343-96f8-3214b8f9f250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:41.494269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ddfe-ccb7-4343-96f8-3214b8f9f250 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.494470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ddfe-ccb7-4343-96f8-3214b8f9f250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.494665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ddfe-ccb7-4343-96f8-3214b8f9f250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.496897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9359ddfe-ccb7-4343-96f8-3214b8f9f250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.544210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 865/1743] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:41.545996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fbc4-2a4b-4ccd-ad43-522940db7e97 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.546704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fbc4-2a4b-4ccd-ad43-522940db7e97 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.547177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fbc4-2a4b-4ccd-ad43-522940db7e97 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.549618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 866/1744] 10.222.0.42 () {54 vars in 912 bytes} [Wed Jul 20 17:49:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:41.551213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d497e-a82b-480b-8d73-58f081d2b3bf None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.551411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d497e-a82b-480b-8d73-58f081d2b3bf None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.551901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d497e-a82b-480b-8d73-58f081d2b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.561271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359ddfe-ccb7-4343-96f8-3214b8f9f250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:41.563490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359ddfe-ccb7-4343-96f8-3214b8f9f250 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.571248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0d3d497e-a82b-480b-8d73-58f081d2b3bf None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:41.572518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0d3d497e-a82b-480b-8d73-58f081d2b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.572634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 879/1745] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:41.573826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738119-6c74-49a2-97f5-4bda34aadac9 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.574066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738119-6c74-49a2-97f5-4bda34aadac9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.574351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738119-6c74-49a2-97f5-4bda34aadac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.581411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 867/1746] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:41.582647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d74b2-7f8f-4e97-8862-d60b175d1379 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.582934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d74b2-7f8f-4e97-8862-d60b175d1379 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.583218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d74b2-7f8f-4e97-8862-d60b175d1379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.593555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-43738119-6c74-49a2-97f5-4bda34aadac9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:41.594940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-43738119-6c74-49a2-97f5-4bda34aadac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.603418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 880/1747] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:41.604715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e82e8-f5b5-4230-9ea8-74627586e466 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.605157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e82e8-f5b5-4230-9ea8-74627586e466 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.605448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e82e8-f5b5-4230-9ea8-74627586e466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.605630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b72d74b2-7f8f-4e97-8862-d60b175d1379 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:41.606833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b72d74b2-7f8f-4e97-8862-d60b175d1379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.619092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 868/1748] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:41.620660 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f157-30c7-4531-987d-a744331ed8a9 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.620961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f157-30c7-4531-987d-a744331ed8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.621238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f157-30c7-4531-987d-a744331ed8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.632997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3c1e82e8-f5b5-4230-9ea8-74627586e466 None None] MFA Rules not processed for user `d1d18a38f45e4501914e22689f145fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:41.635685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3c1e82e8-f5b5-4230-9ea8-74627586e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.653692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7e50f157-30c7-4531-987d-a744331ed8a9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:41.654907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7e50f157-30c7-4531-987d-a744331ed8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.663685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 869/1749] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:41.664983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb2fc-d8ca-4b65-99b3-f47d3d5a8feb None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.665272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb2fc-d8ca-4b65-99b3-f47d3d5a8feb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.665549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb2fc-d8ca-4b65-99b3-f47d3d5a8feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.689840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-260cb2fc-d8ca-4b65-99b3-f47d3d5a8feb None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:41.691193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-260cb2fc-d8ca-4b65-99b3-f47d3d5a8feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.700146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 870/1750] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:41.700868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb2fc-d8ca-4b65-99b3-f47d3d5a8feb None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.710914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb2fc-d8ca-4b65-99b3-f47d3d5a8feb None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:41.716806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:41.717476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.718007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.718007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.719441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:41.719753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:41.720622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 881/1751] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:41.721817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e82e8-f5b5-4230-9ea8-74627586e466 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.722330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.735641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e82e8-f5b5-4230-9ea8-74627586e466 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:41.746700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:41.747528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.747815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.748205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.748322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 871/1752] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:41.748914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:41.749225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:41.751598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.753124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.762984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79039b6f-2796-42a2-98ee-ec3655333c10 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:41.765862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 882/1753] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:41.770915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.776106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:41.776515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:41.776612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.776612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.776612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.776612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:41.776612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:41.777376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.786684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606fe9f-1a0d-46fc-b039-1e3c3d75e959 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:41.791078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:41.791442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 872/1754] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:41.791876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.792197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.792499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.792662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.802998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea2b4-f1d6-4868-9d16-334b2573ed6b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:41.804872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:41.805963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.808174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:41.808957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.809308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.809657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.810430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:41.812462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.820875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 883/1755] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:41.821681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.853302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b3713-8c0d-4e60-88f4-c4b02fc6c6af service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:41.860163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:41.865179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:41.865179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:41.865179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:41.865179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:41.865179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:41.957981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 873/1756] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:41.958680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:41.986129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 884/1757] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:41.987317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.032002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e88e9-1dd6-4b26-aad4-7c7486dc41f8 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.092412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.093481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.094009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.094499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.095419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.097610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.118645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e2f7d-0015-40f4-99af-7c531c8ee6d2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.129484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.132169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.132515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.132807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.133578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.135713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.291937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 885/1758] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:42.293054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.297093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 874/1759] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:41 2022] POST /identity/v3/projects => generated 373 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:42.297093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.299004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:42.420426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e59ad-17f6-4582-bea9-ecb494fa95c4 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.449553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986ed5e-6021-404a-93f0-9ba04a84c093 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.462972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.464240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.464687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.465124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.466965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.471827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.563221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.563882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.565359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.565359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.567156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:42.588620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 875/1760] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:42 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:42.588620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.611196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff328b30-0c09-4a0a-bfb6-0bea2b0f44a8 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.651290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.651993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.652282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.652620 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.653369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.656769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.661601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.663408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.663803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:42.678755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 886/1761] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:42.680183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.695161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ec9d3-33a8-44cd-bbda-f1a67559d8b5 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.702462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.704076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.704518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.704818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.705583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.707794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.712961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 876/1762] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:42.715552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.725497 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70212-3334-4d3d-90fc-35fb60281162 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.730182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.730855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.731138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.731416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.732154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.733790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.745290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 877/1763] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:42.746201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.756794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af20f6-11b9-4b93-a15a-07906ed2d2d6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.769873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.769873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.769873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.769873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.770217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.770217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.779407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 878/1764] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:42.779407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.786302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:42.788844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e900af-2864-4142-9125-45e95fbfa5fd admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.793481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.794132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.794418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.794695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.795419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.797573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.821764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 887/1765] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:42 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:42.821764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.822256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 879/1766] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:42.823143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.839869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232203ce-4450-47ea-a706-df1581435507 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.840255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef5ee3-4788-4ea8-88f4-1cf74f0607e6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.848202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.848934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4a5d2-942f-4d2c-b6f0-ffa0156208d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.849096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.849215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.849597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.850257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.850444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4a5d2-942f-4d2c-b6f0-ffa0156208d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.850907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4a5d2-942f-4d2c-b6f0-ffa0156208d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.851292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4a5d2-942f-4d2c-b6f0-ffa0156208d6 admin admin] PATH_INFO: `/v3/projects/9c55263b75d1457690f9c5e5f04124a8/users/349efd8ac7e94760b00be3216c970704/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.852050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.870400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 880/1767] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:49:42.871394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:42.880325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4a5d2-942f-4d2c-b6f0-ffa0156208d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c55263b75d1457690f9c5e5f04124a8, user_id=349efd8ac7e94760b00be3216c970704, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.883495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4a5d2-942f-4d2c-b6f0-ffa0156208d6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:42.886305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ff6dd-8179-4242-a69c-087519a9cacf admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:42.891857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:42.892787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:42.893083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:42.893340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] PATH_INFO: `/v3/projects/47377dd50c7043f588abf5daea55b256/users/b7d9eaae3f9d4c72be2d57411c60451c/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:42.909834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47377dd50c7043f588abf5daea55b256, user_id=b7d9eaae3f9d4c72be2d57411c60451c, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:42.912550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:43.023253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 881/1768] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:42 2022] PUT /identity/v3/projects/47377dd50c7043f588abf5daea55b256/users/b7d9eaae3f9d4c72be2d57411c60451c/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:43.025369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:43.036696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 888/1769] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:42 2022] PUT /identity/v3/projects/9c55263b75d1457690f9c5e5f04124a8/users/349efd8ac7e94760b00be3216c970704/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:43.038099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ba4a0-2449-4c38-aed9-b7f22f1480c2 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.038567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ba4a0-2449-4c38-aed9-b7f22f1480c2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.039540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ba4a0-2449-4c38-aed9-b7f22f1480c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.067437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c39ba4a0-2449-4c38-aed9-b7f22f1480c2 None None] MFA Rules not processed for user `b7d9eaae3f9d4c72be2d57411c60451c`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:43.068802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c39ba4a0-2449-4c38-aed9-b7f22f1480c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:43.096526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa0aa-338e-4043-ac03-35c1fad17292 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:43.107466 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de344d5-e68b-4805-969d-2fe68d9780aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:43.109528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de344d5-e68b-4805-969d-2fe68d9780aa admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.109977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de344d5-e68b-4805-969d-2fe68d9780aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.110428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de344d5-e68b-4805-969d-2fe68d9780aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.111328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de344d5-e68b-4805-969d-2fe68d9780aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:43.118845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de344d5-e68b-4805-969d-2fe68d9780aa admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:43.131517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 882/1770] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:43.134715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638de3b-a2fa-4ebd-9c35-230d21fe27c8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.135161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638de3b-a2fa-4ebd-9c35-230d21fe27c8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.135749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638de3b-a2fa-4ebd-9c35-230d21fe27c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.154406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 889/1771] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:43.160639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ba4a0-2449-4c38-aed9-b7f22f1480c2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:43.177552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ba4a0-2449-4c38-aed9-b7f22f1480c2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:43.179670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b638de3b-a2fa-4ebd-9c35-230d21fe27c8 None None] MFA Rules not processed for user `349efd8ac7e94760b00be3216c970704`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:43.181067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b638de3b-a2fa-4ebd-9c35-230d21fe27c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:43.184109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:43.185641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.186047 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.186929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] PATH_INFO: `/v3/projects/a2143ad19e02482c93b96b20660bee5a/users/04fb23329bc8406c9198d16e26d492dc/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.212901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2143ad19e02482c93b96b20660bee5a, user_id=04fb23329bc8406c9198d16e26d492dc, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:43.215943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:43.266360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 883/1772] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:43.267232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638de3b-a2fa-4ebd-9c35-230d21fe27c8 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:43.278592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638de3b-a2fa-4ebd-9c35-230d21fe27c8 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:43.283725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10614b-0e7b-460b-abfb-49ba450d51d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:43.284374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10614b-0e7b-460b-abfb-49ba450d51d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.284879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10614b-0e7b-460b-abfb-49ba450d51d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.285180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10614b-0e7b-460b-abfb-49ba450d51d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.285943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10614b-0e7b-460b-abfb-49ba450d51d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:43.288949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10614b-0e7b-460b-abfb-49ba450d51d4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:43.360710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 890/1773] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:43 2022] PUT /identity/v3/projects/a2143ad19e02482c93b96b20660bee5a/users/04fb23329bc8406c9198d16e26d492dc/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:43.361978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:43.367060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:43.440492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa04031-419c-41dd-ac84-a695f86ea3e1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:43.444173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:43.444859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.445145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.445429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.484582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 884/1774] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:43 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:43.484582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53316963-c728-498b-8281-2ecf177f31d2 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.484582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53316963-c728-498b-8281-2ecf177f31d2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.484582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53316963-c728-498b-8281-2ecf177f31d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.511710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:43.514490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:43.561445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-53316963-c728-498b-8281-2ecf177f31d2 None None] MFA Rules not processed for user `04fb23329bc8406c9198d16e26d492dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:43.561445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-53316963-c728-498b-8281-2ecf177f31d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:43.561701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 891/1775] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:43.567017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:43.719815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 885/1776] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:43.722253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53316963-c728-498b-8281-2ecf177f31d2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:43.728383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be943c3-ee74-4f4d-90c1-0236e3a92198 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:43.834327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53316963-c728-498b-8281-2ecf177f31d2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:43.836739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:43.837516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.837813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.838179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.839099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:43.840160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:43.840996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:43.841456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:43.841570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:43.841848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:43.842610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:43.845790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.003611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 892/1777] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:43 2022] POST /identity/v3/users => generated 298 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:44.003804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 886/1778] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:43 2022] POST /identity/v3/projects => generated 371 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:44.006611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.006753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.017410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50cdc4-7ee0-4e99-9668-671976650905 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.070614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a82c0-2dba-4f5b-9a01-93ce93f80988 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.073412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.074311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.074770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.075053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.075786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.075951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.076719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.077013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.077299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.077454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.078601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.080576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.087860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 893/1779] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:44.091963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.102113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e003c-165d-4586-93fd-c4169eb55d29 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.109005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.109892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.110178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.110458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] PATH_INFO: `/v3/projects/636c83e93f4942edbd363f7ba359e5ce/users/b41d54ef5d984c15bfb4fb337cd575d7/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.125382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636c83e93f4942edbd363f7ba359e5ce, user_id=b41d54ef5d984c15bfb4fb337cd575d7, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.128140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.227506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 887/1780] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:44 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:44.228776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.238585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06d4fa-fc3c-4e5c-8bbb-f857eaa01ecd admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.243293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.243951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.244238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.244632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.245371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.246985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.269916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 888/1781] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:49:44.271363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 894/1782] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:44 2022] PUT /identity/v3/projects/636c83e93f4942edbd363f7ba359e5ce/users/b41d54ef5d984c15bfb4fb337cd575d7/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:49:44.272906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.275654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.285846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdbecc-d190-44aa-8159-85b50a64c882 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.356302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.357225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.357508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.357789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] PATH_INFO: `/v3/projects/660b25d695ac4fa2bf802f6cdfd6af79/users/5b9967c48c104252b6bf186753b87620/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.360606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288414f-4c3d-4830-914e-860b0c18e6b9 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.365367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.366190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.366528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.366815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.367553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.369224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.394537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 895/1783] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:49:44.397269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] RBAC: Authorizing `identity:create_grant(project_id=660b25d695ac4fa2bf802f6cdfd6af79, user_id=5b9967c48c104252b6bf186753b87620, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.400417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.415911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.417125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11d514-279a-4a89-9988-937a2cf74a80 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.423040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c99c4-6adb-4fe4-8f67-c47447e6cc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.425197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c99c4-6adb-4fe4-8f67-c47447e6cc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.428656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c99c4-6adb-4fe4-8f67-c47447e6cc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.428656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c99c4-6adb-4fe4-8f67-c47447e6cc78 admin admin] PATH_INFO: `/v3/projects/636c83e93f4942edbd363f7ba359e5ce/users/b41d54ef5d984c15bfb4fb337cd575d7/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.429398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c99c4-6adb-4fe4-8f67-c47447e6cc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636c83e93f4942edbd363f7ba359e5ce, user_id=b41d54ef5d984c15bfb4fb337cd575d7, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.432887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c99c4-6adb-4fe4-8f67-c47447e6cc78 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.457475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 896/1784] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:44 2022] PUT /identity/v3/projects/636c83e93f4942edbd363f7ba359e5ce/users/b41d54ef5d984c15bfb4fb337cd575d7/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:44.459694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012b65e-b1a4-4b84-b2a1-2466075f19f4 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.460014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012b65e-b1a4-4b84-b2a1-2466075f19f4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.460273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012b65e-b1a4-4b84-b2a1-2466075f19f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.497809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0012b65e-b1a4-4b84-b2a1-2466075f19f4 None None] MFA Rules not processed for user `b41d54ef5d984c15bfb4fb337cd575d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:44.499103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0012b65e-b1a4-4b84-b2a1-2466075f19f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:44.521401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 889/1785] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:44 2022] PUT /identity/v3/projects/660b25d695ac4fa2bf802f6cdfd6af79/users/5b9967c48c104252b6bf186753b87620/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:49:44.522932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.577635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 897/1786] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:44.589496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012b65e-b1a4-4b84-b2a1-2466075f19f4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.676346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f3216-0244-4745-b095-f2a391b9636a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.680009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012b65e-b1a4-4b84-b2a1-2466075f19f4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.688298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.689047 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.689610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.689901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.690650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.692278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.708081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:44.724845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 890/1787] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:44.726286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:44.746713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a2c7a-d54a-4162-8655-a08566026f73 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:44.760602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8063403-2aa4-4821-9367-4f2a2cbdd433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.761720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8063403-2aa4-4821-9367-4f2a2cbdd433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.763163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8063403-2aa4-4821-9367-4f2a2cbdd433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.763163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8063403-2aa4-4821-9367-4f2a2cbdd433 admin admin] PATH_INFO: `/v3/projects/660b25d695ac4fa2bf802f6cdfd6af79/users/5b9967c48c104252b6bf186753b87620/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.767411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8063403-2aa4-4821-9367-4f2a2cbdd433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=660b25d695ac4fa2bf802f6cdfd6af79, user_id=5b9967c48c104252b6bf186753b87620, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.774678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8063403-2aa4-4821-9367-4f2a2cbdd433 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.789609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:44.794327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:44.795150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.795400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.805862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.806025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 891/1788] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:44 2022] PUT /identity/v3/projects/660b25d695ac4fa2bf802f6cdfd6af79/users/5b9967c48c104252b6bf186753b87620/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:49:44.814726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc2ddc-963c-4db2-abe7-51b7125fdd1f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:44.817382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc2ddc-963c-4db2-abe7-51b7125fdd1f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:44.818475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc2ddc-963c-4db2-abe7-51b7125fdd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:44.875751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-99dc2ddc-963c-4db2-abe7-51b7125fdd1f None None] MFA Rules not processed for user `5b9967c48c104252b6bf186753b87620`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:44.877647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-99dc2ddc-963c-4db2-abe7-51b7125fdd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:44.892641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:44.895282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:44.908180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 898/1789] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:44.988519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 892/1790] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:45.028124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:45.046867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed365823-03f4-4f53-bcf1-399a854bd092 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:45.100302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:45.101223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:45.102692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:45.102692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:45.117978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:45.119444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:45.131463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 899/1791] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:45.190974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc2ddc-963c-4db2-abe7-51b7125fdd1f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:45.200523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc2ddc-963c-4db2-abe7-51b7125fdd1f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:45.204874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8b98-c6cb-47be-8994-8750b18768e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:45.205526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8b98-c6cb-47be-8994-8750b18768e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:45.207231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8b98-c6cb-47be-8994-8750b18768e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:45.207231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8b98-c6cb-47be-8994-8750b18768e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:45.264162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8b98-c6cb-47be-8994-8750b18768e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:45.265758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8b98-c6cb-47be-8994-8750b18768e7 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:45.275109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 893/1792] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:46.222035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:46.874573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:46.937798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7435c-a211-4ad1-8cfa-2c824c8652c0 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:46.942400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:46.943296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:46.943579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:46.943859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] PATH_INFO: `/v3/users/04fb23329bc8406c9198d16e26d492dc` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:46.945062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04fb23329bc8406c9198d16e26d492dc)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:46.947194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:47.259681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04fb23329bc8406c9198d16e26d492dc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:47.260343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04fb23329bc8406c9198d16e26d492dc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:47.260990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04fb23329bc8406c9198d16e26d492dc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:47.267740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04fb23329bc8406c9198d16e26d492dc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:47.371831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04fb23329bc8406c9198d16e26d492dc'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:47.391942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 900/1793] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:46 2022] DELETE /identity/v3/users/04fb23329bc8406c9198d16e26d492dc => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:47.396513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c0db3-1fe1-4f0c-9a6a-28d35d3ca609 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:47.397013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c0db3-1fe1-4f0c-9a6a-28d35d3ca609 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:47.397100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c0db3-1fe1-4f0c-9a6a-28d35d3ca609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:47.415056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4e6c0db3-1fe1-4f0c-9a6a-28d35d3ca609 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:47.416264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4e6c0db3-1fe1-4f0c-9a6a-28d35d3ca609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:47.463655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:INSERT=4,keystone:SELECT=95,keystone:DELETE=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:47.479809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 894/1794] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:47.489842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:47.491286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:47.558244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c2122-dd8d-4dc8-80ea-dc7ce1684244 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:47.562864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:47.563679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:47.564105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:47.564517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:47.574720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:47.576183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:47.587021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 901/1795] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:47.610450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270aa8c-8e7d-4b79-ab88-9c4759e6e19d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:47.610858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270aa8c-8e7d-4b79-ab88-9c4759e6e19d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:47.611211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270aa8c-8e7d-4b79-ab88-9c4759e6e19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:47.630746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4270aa8c-8e7d-4b79-ab88-9c4759e6e19d None None] MFA Rules not processed for user `d1d18a38f45e4501914e22689f145fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:47.632056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4270aa8c-8e7d-4b79-ab88-9c4759e6e19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:47.641718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 895/1796] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jul 20 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:47.762797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:47.774607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495eff0-2c93-48f3-9735-12670406ed29 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:47.778884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:47.780079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:47.780444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:47.780867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:47.790765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:47.792292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:47.805304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 902/1797] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:49:47.826272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393002-cca1-47a7-b914-24843f460ecd None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:47.826641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393002-cca1-47a7-b914-24843f460ecd None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:47.826928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393002-cca1-47a7-b914-24843f460ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:47.882359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-96393002-cca1-47a7-b914-24843f460ecd None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:47.883815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-96393002-cca1-47a7-b914-24843f460ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:47.972075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 896/1798] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:49:48.210699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:48.213111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:48.231359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bcd57-917f-4889-a248-95f25ed9f281 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:48.237136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:48.238348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:48.238691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:48.239050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] PATH_INFO: `/v3/projects/a2143ad19e02482c93b96b20660bee5a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:48.240418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2143ad19e02482c93b96b20660bee5a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:48.265829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:48.311564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393002-cca1-47a7-b914-24843f460ecd None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:48.313637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-96393002-cca1-47a7-b914-24843f460ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:48.417002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2143ad19e02482c93b96b20660bee5a', 'request_id': 'req-72e60663-6de8-4235-91c0-27ef722e4217'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:48.427179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2143ad19e02482c93b96b20660bee5a', 'request_id': 'req-72e60663-6de8-4235-91c0-27ef722e4217'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:48.433390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393002-cca1-47a7-b914-24843f460ecd None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:48.497302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2143ad19e02482c93b96b20660bee5a', 'request_id': 'req-72e60663-6de8-4235-91c0-27ef722e4217'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:48.522506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] The token cache is being invalidate because project a2143ad19e02482c93b96b20660bee5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:48.522700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:48.524085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:48.524912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:48.526180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:48.526736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:48.527350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:48.529182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 903/1799] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:48 2022] DELETE /identity/v3/projects/a2143ad19e02482c93b96b20660bee5a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:48.529810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:48.532296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:48.532817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:48.637973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:48.642262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:48.652927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 897/1800] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:49:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 17:49:48.660505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:48.660505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:48.669544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e60663-6de8-4235-91c0-27ef722e4217 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:48.681012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:48.683084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:48.683084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:48.683084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] PATH_INFO: `/v3/users/5b9967c48c104252b6bf186753b87620` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:48.705114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b9967c48c104252b6bf186753b87620)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:48.707139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:48.770511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763f93e-8a75-4092-9a85-f6f6e60091b3 service swift] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:48.775986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:48.776776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:48.777604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:48.777604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:48.782826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:48.809602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:48.929412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:48.929412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:48.980755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 898/1801] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:48.996516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b9967c48c104252b6bf186753b87620'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:48.997705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b9967c48c104252b6bf186753b87620'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:49.001385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b9967c48c104252b6bf186753b87620'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:49.013373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b9967c48c104252b6bf186753b87620'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:49.070854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-566821cc-a536-46d1-8a5a-6edc1ffbbf3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b9967c48c104252b6bf186753b87620'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:49.148122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 904/1802] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:48 2022] DELETE /identity/v3/users/5b9967c48c104252b6bf186753b87620 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:49.412872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:50.572672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:50.603384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3498d-b3f2-44da-a7eb-3013f41f3391 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:50.786197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:50.790181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:50.790709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:50.791627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] PATH_INFO: `/v3/projects/660b25d695ac4fa2bf802f6cdfd6af79` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:50.801731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=660b25d695ac4fa2bf802f6cdfd6af79)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:50.818897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:51.002216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '660b25d695ac4fa2bf802f6cdfd6af79', 'request_id': 'req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:51.013794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '660b25d695ac4fa2bf802f6cdfd6af79', 'request_id': 'req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:51.018917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986c86-64e8-469a-a047-9d39ad8ebb3c None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:51.021404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986c86-64e8-469a-a047-9d39ad8ebb3c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:51.021404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986c86-64e8-469a-a047-9d39ad8ebb3c None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:51.022625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 905/1803] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:49:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:51.044360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '660b25d695ac4fa2bf802f6cdfd6af79', 'request_id': 'req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:51.055430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce95854-de94-496b-9d3a-785aa3adfc0e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:51.055798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce95854-de94-496b-9d3a-785aa3adfc0e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:51.056125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce95854-de94-496b-9d3a-785aa3adfc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:51.102798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:51.105491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0ce95854-de94-496b-9d3a-785aa3adfc0e None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:51.111174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0ce95854-de94-496b-9d3a-785aa3adfc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:51.178567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] The token cache is being invalidate because project 660b25d695ac4fa2bf802f6cdfd6af79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:51.178924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:51.187898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e79ed0ee-ac5f-4ef6-a935-f55b2392fafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:51.198022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 899/1804] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:50 2022] DELETE /identity/v3/projects/660b25d695ac4fa2bf802f6cdfd6af79 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:51.219742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39e6d-023b-47a2-af8c-ff973dce73e5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:51.219742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39e6d-023b-47a2-af8c-ff973dce73e5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:51.219742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39e6d-023b-47a2-af8c-ff973dce73e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:51.267609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e5b39e6d-023b-47a2-af8c-ff973dce73e5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:51.270736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e5b39e6d-023b-47a2-af8c-ff973dce73e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:51.371168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 906/1805] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:49:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:51.539155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 900/1806] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:51.549735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827baa-5b95-44d3-a0cc-509f1d40fe89 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:51.550036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827baa-5b95-44d3-a0cc-509f1d40fe89 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:51.550425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827baa-5b95-44d3-a0cc-509f1d40fe89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:51.600851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-8e827baa-5b95-44d3-a0cc-509f1d40fe89 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:51.605753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8e827baa-5b95-44d3-a0cc-509f1d40fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:51.639779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 907/1807] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:51.645375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39e6d-023b-47a2-af8c-ff973dce73e5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:51.665192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39e6d-023b-47a2-af8c-ff973dce73e5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:51.677164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:51.678127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:51.678565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:51.679008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:51.679976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:49:51.680297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:51.682813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:51.711048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 901/1808] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:49:51.715992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827baa-5b95-44d3-a0cc-509f1d40fe89 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:51.733050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827baa-5b95-44d3-a0cc-509f1d40fe89 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:51.744826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:51.745943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:51.746392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:51.746878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:51.748223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:51.750903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:51.898318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 908/1809] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:51 2022] POST /identity/v3/projects => generated 357 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:51.898491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:51.929693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8cd9d-b67a-4a44-92de-96d4eaa74a45 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:52.084925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:52.089057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:52.089057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:52.089057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:52.089057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:52.089057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:52.211540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 902/1810] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:51 2022] POST /identity/v3/users => generated 279 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:49:52.218163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:52.238369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:52.254439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448a205-5c07-487d-be16-649708bce175 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:52.268708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:52.269684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:52.270129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:52.270612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:52.271527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:52.277592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:52.311942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 909/1811] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:52.321626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:52.357938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47bbd4-fe7f-4130-849a-36eb5c8dbde3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:52.378019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:52.384604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:52.384604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:52.384604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:52.384604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:52.384604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:52.403560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 903/1812] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:52.410358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:52.433706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc6fa3-7aab-4aeb-a667-dd87b4044182 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:52.460704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3bc5a-5659-40ec-b580-b5d5be824d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:52.461592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3bc5a-5659-40ec-b580-b5d5be824d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:52.461858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3bc5a-5659-40ec-b580-b5d5be824d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:52.462144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3bc5a-5659-40ec-b580-b5d5be824d1b admin admin] PATH_INFO: `/v3/projects/ec32a5930db347bea1ec76fa7e25794c/users/c8d90ec4f0c240edba386f248ec8906c/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:52.487182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3bc5a-5659-40ec-b580-b5d5be824d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec32a5930db347bea1ec76fa7e25794c, user_id=c8d90ec4f0c240edba386f248ec8906c, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:52.512123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3bc5a-5659-40ec-b580-b5d5be824d1b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:52.546614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:52.549171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:52.619643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 910/1813] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:52 2022] PUT /identity/v3/projects/ec32a5930db347bea1ec76fa7e25794c/users/c8d90ec4f0c240edba386f248ec8906c/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:52.638571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e9bf7-cd65-4b34-a39c-26a5ce5e90e3 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:52.642024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e9bf7-cd65-4b34-a39c-26a5ce5e90e3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:52.642339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e9bf7-cd65-4b34-a39c-26a5ce5e90e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:52.775084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e42e9bf7-cd65-4b34-a39c-26a5ce5e90e3 None None] MFA Rules not processed for user `c8d90ec4f0c240edba386f248ec8906c`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:52.776376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e42e9bf7-cd65-4b34-a39c-26a5ce5e90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:52.867179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800b66-2c47-4cd9-9c0d-728ad54e3486 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:52.937418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 911/1814] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:52 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:49:52.949311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e9bf7-cd65-4b34-a39c-26a5ce5e90e3 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:53.026932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:53.109856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:53.110550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.110840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.111125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.119248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e9bf7-cd65-4b34-a39c-26a5ce5e90e3 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:53.137036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:53.138820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:53.145922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.145922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.145922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.145922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:53.149187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:53.151564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:53.185816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 904/1815] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:49:53.262203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 912/1816] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:49:52 2022] POST /identity/v3/projects => generated 353 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:53.268342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:53.280050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a5715-4c62-4685-8b20-fa26e30ed624 service nova] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:53.347090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:53.349515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:53.353301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:53.354168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.354457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.354735 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.355437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:53.357453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:53.412228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf12af-9c05-44f9-a5d0-ddb1784c64a0 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:53.417383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474869bf-7129-43cd-9ace-1876cc60277b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:53.419478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474869bf-7129-43cd-9ace-1876cc60277b service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.419478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474869bf-7129-43cd-9ace-1876cc60277b service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.419478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474869bf-7129-43cd-9ace-1876cc60277b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.420947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-474869bf-7129-43cd-9ace-1876cc60277b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:53.447542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 905/1817] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:49:53 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:49:53.451153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:53.475013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd5206-3f66-4c2a-a490-7ea01125e4f2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:53.481961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74934bc4-1733-4f96-9898-79e833f1fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:53.490827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74934bc4-1733-4f96-9898-79e833f1fc02 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.491352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74934bc4-1733-4f96-9898-79e833f1fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.491711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74934bc4-1733-4f96-9898-79e833f1fc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.492976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74934bc4-1733-4f96-9898-79e833f1fc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:53.497674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74934bc4-1733-4f96-9898-79e833f1fc02 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:53.532835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 906/1818] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:53.535060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81388-3e4d-41f3-9967-3309df3298f6 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.535342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81388-3e4d-41f3-9967-3309df3298f6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.535640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81388-3e4d-41f3-9967-3309df3298f6 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.537034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 907/1819] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:49:53.537740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c81388-3e4d-41f3-9967-3309df3298f6 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:53.596024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c81388-3e4d-41f3-9967-3309df3298f6 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:53.597908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474869bf-7129-43cd-9ace-1876cc60277b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:53.601687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:53.602004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474869bf-7129-43cd-9ace-1876cc60277b service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:53.606551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.606841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.607121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] PATH_INFO: `/v3/projects/5f1be7a73dc341c98eed3bac9ccd18ba/users/27adce2541684ca08aa466d1c2a9497b/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.625490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 913/1820] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:53.629794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d9fac-bc9a-432a-b4e7-28b7b2f8c6c8 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:53.630255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d9fac-bc9a-432a-b4e7-28b7b2f8c6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:53.631092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d9fac-bc9a-432a-b4e7-28b7b2f8c6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:53.701666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3a0d9fac-bc9a-432a-b4e7-28b7b2f8c6c8 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:53.703636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3a0d9fac-bc9a-432a-b4e7-28b7b2f8c6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:53.716968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:53.721402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1be7a73dc341c98eed3bac9ccd18ba, user_id=27adce2541684ca08aa466d1c2a9497b, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:53.730696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:53.830233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:53.842163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 908/1821] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:53 2022] PUT /identity/v3/projects/5f1be7a73dc341c98eed3bac9ccd18ba/users/27adce2541684ca08aa466d1c2a9497b/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:53.850918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:53.954738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 914/1822] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:49:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:54.019055 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2623c-10d6-4735-9383-4a1db9f513b3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:54.033636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8980aae-0a2c-411d-b878-681c6e82dd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:54.035814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8980aae-0a2c-411d-b878-681c6e82dd06 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:54.036320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8980aae-0a2c-411d-b878-681c6e82dd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:54.036782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8980aae-0a2c-411d-b878-681c6e82dd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:54.037720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8980aae-0a2c-411d-b878-681c6e82dd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:54.039657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8980aae-0a2c-411d-b878-681c6e82dd06 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:54.103777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 909/1823] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:49:54.108530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d9fac-bc9a-432a-b4e7-28b7b2f8c6c8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:54.120970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d9fac-bc9a-432a-b4e7-28b7b2f8c6c8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:54.129273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:54.156364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:54.156986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:54.157468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] PATH_INFO: `/v3/projects/5f1be7a73dc341c98eed3bac9ccd18ba/users/27adce2541684ca08aa466d1c2a9497b/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:54.163244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1be7a73dc341c98eed3bac9ccd18ba, user_id=27adce2541684ca08aa466d1c2a9497b, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:54.169043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:54.184476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 915/1824] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:49:54 2022] PUT /identity/v3/projects/5f1be7a73dc341c98eed3bac9ccd18ba/users/27adce2541684ca08aa466d1c2a9497b/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:49:54.198416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:54.200822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:54.200822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:54.235566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] MFA Rules not processed for user `27adce2541684ca08aa466d1c2a9497b`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:54.237054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:54.265592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:54.269658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:54.334136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 910/1825] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:54 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:54.350019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:54.351492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:54.384284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600a1a8-f104-46fc-9dcc-4cd2e61c3f4e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:54.391298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:54.391985 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:54.392323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:54.392711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:54.402832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:54.404654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:54.429464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 916/1826] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:54.442212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25831267-e426-475a-86b4-9b7ab49ad821 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:54.454267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:54.454953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:54.460008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:54.460008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:54.476632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:54.478441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:54.493338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 911/1827] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:54 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:49:54.746358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:54.758348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c41fbb-356d-43b2-ad18-005a72298c4c service nova] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:54.763244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19b6fb-699e-4d2f-aa7c-79dc62b89289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:54.763895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19b6fb-699e-4d2f-aa7c-79dc62b89289 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:54.764181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19b6fb-699e-4d2f-aa7c-79dc62b89289 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:54.764529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19b6fb-699e-4d2f-aa7c-79dc62b89289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:54.821419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19b6fb-699e-4d2f-aa7c-79dc62b89289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:54.822819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19b6fb-699e-4d2f-aa7c-79dc62b89289 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:54.839539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 917/1828] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:54 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:57.644086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:57.645624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:57.731813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460812b-890a-4702-8930-273eedbcf96d service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:57.739415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:57.740416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:57.740820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:57.741274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] PATH_INFO: `/v3/users/b7d9eaae3f9d4c72be2d57411c60451c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:57.742636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d9eaae3f9d4c72be2d57411c60451c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:57.744999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:57.977034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d9eaae3f9d4c72be2d57411c60451c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:57.994046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d9eaae3f9d4c72be2d57411c60451c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:57.994811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d9eaae3f9d4c72be2d57411c60451c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:57.995509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d9eaae3f9d4c72be2d57411c60451c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:58.022855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d9eaae3f9d4c72be2d57411c60451c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:58.131971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:58.131971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:58.132971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:58.158666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 912/1829] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:57 2022] DELETE /identity/v3/users/b7d9eaae3f9d4c72be2d57411c60451c => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:58.184562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:49:58.185896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:58.213178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:58.282164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:49:58.305601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 918/1830] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:49:58.321888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:58.325693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:58.466870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808467a2-1bbc-4f1e-be9b-e58ee3c83ee1 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:58.504876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:58.505747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:58.506199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:58.506662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:58.538985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:58.542005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:58.568861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 913/1831] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:49:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:49:59.257969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:59.259633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:49:59.285252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5933d-13c8-44d1-bafe-76d96a13986d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:59.295535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:59.297015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:59.299924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:59.299924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] PATH_INFO: `/v3/projects/47377dd50c7043f588abf5daea55b256` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:59.302209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] RBAC: Authorizing `identity:delete_project(project_id=47377dd50c7043f588abf5daea55b256)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:59.304367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:59.460328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47377dd50c7043f588abf5daea55b256', 'request_id': 'req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.473250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47377dd50c7043f588abf5daea55b256', 'request_id': 'req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.548729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47377dd50c7043f588abf5daea55b256', 'request_id': 'req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.577221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] The token cache is being invalidate because project 47377dd50c7043f588abf5daea55b256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:49:59.577526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.580245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.583136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 919/1832] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:49:59 2022] DELETE /identity/v3/projects/47377dd50c7043f588abf5daea55b256 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:49:59.587219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:49:59.601249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310c2aa-7f8a-4aa0-81a6-4f8d6ee03acc service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:49:59.707369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:49:59.708258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:49:59.709899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:49:59.709899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] PATH_INFO: `/v3/users/b41d54ef5d984c15bfb4fb337cd575d7` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:49:59.710140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41d54ef5d984c15bfb4fb337cd575d7)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:49:59.712062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:49:59.931171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41d54ef5d984c15bfb4fb337cd575d7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.951057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41d54ef5d984c15bfb4fb337cd575d7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.951785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41d54ef5d984c15bfb4fb337cd575d7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.952968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41d54ef5d984c15bfb4fb337cd575d7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:49:59.973363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41d54ef5d984c15bfb4fb337cd575d7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:00.141816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 914/1833] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:49:59 2022] DELETE /identity/v3/users/b41d54ef5d984c15bfb4fb337cd575d7 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:00.558711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:00.826397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:00.828517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:00.846154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04c4fa-3f86-46e5-a9e2-2300b7caf31d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:00.881161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:00.882812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:00.920452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:00.921677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:00.922119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:00.924692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] PATH_INFO: `/v3/projects/636c83e93f4942edbd363f7ba359e5ce` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:00.926685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=636c83e93f4942edbd363f7ba359e5ce)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:00.929163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:00.976564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d063ce-9907-4e32-9c25-18078a18f727 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:00.981937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:00.984571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:00.984571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:00.984571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] PATH_INFO: `/v3/users/8c22854b3b234d879f233e040c8662b2` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:00.985181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c22854b3b234d879f233e040c8662b2)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:00.987162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.056920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636c83e93f4942edbd363f7ba359e5ce', 'request_id': 'req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.064566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636c83e93f4942edbd363f7ba359e5ce', 'request_id': 'req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.169858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636c83e93f4942edbd363f7ba359e5ce', 'request_id': 'req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.180790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] The token cache is being invalidate because project 636c83e93f4942edbd363f7ba359e5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:01.181022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.181667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8f55bfa9-27f9-4fb8-9a88-b3c14679f80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.183490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 920/1834] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:00 2022] DELETE /identity/v3/projects/636c83e93f4942edbd363f7ba359e5ce => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:01.188948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ed8e-6ca4-407a-bfc2-bbde38cda0be None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.189340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ed8e-6ca4-407a-bfc2-bbde38cda0be None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.189687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049ed8e-6ca4-407a-bfc2-bbde38cda0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.208367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-2049ed8e-6ca4-407a-bfc2-bbde38cda0be None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:01.209582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2049ed8e-6ca4-407a-bfc2-bbde38cda0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:01.239572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c22854b3b234d879f233e040c8662b2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.254021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c22854b3b234d879f233e040c8662b2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.254759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c22854b3b234d879f233e040c8662b2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.255636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c22854b3b234d879f233e040c8662b2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.258069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:01.263349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a1a3a774-2c93-4b82-aac8-aeb377d16ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c22854b3b234d879f233e040c8662b2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:01.275896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 921/1835] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:01.284577 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e2101-cd69-4b3a-90d8-4e7c3ea59649 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.288355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e2101-cd69-4b3a-90d8-4e7c3ea59649 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.288355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e2101-cd69-4b3a-90d8-4e7c3ea59649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.303928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-843e2101-cd69-4b3a-90d8-4e7c3ea59649 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:01.305199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-843e2101-cd69-4b3a-90d8-4e7c3ea59649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:01.313859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 922/1836] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:01.321525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e2101-cd69-4b3a-90d8-4e7c3ea59649 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:01.330886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e2101-cd69-4b3a-90d8-4e7c3ea59649 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:01.337040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:01.341324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.341324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.341324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.341324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:01.341324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:01.341324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.341869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 915/1837] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:00 2022] DELETE /identity/v3/users/8c22854b3b234d879f233e040c8662b2 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:01.345748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.346234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.346669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.354952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 923/1838] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:50:01.359753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:01.365370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:01.369943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:01.370064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5b1c7-811e-450e-9bc0-96b6d0060ed1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:01.425328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:01.426001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.426293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.426575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.427281 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:01.427808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 916/1839] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:01.429462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.438033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:01.439799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:01.494103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ff1b2-5a35-477a-95a0-56a1cd7bdacf None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:01.497784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:01.498423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.498706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.498987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.508914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:01.510364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.521954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 924/1841] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:01 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:01.524963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 917/1841] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:01.525955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:01.585966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac757bdd-eb27-48b5-b549-3f4f6b35a965 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:01.590766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:01.593758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.593758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.593758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.593758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:01.594725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.677971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 925/1842] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:01 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:01.679279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:01.689035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0fad-a4d0-433a-8d64-3912634b6adf service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:01.693556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:01.694219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.694503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.694781 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.695549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:01.697216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.707087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 918/1843] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:01.718600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:01.729792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03edad-7234-401a-bcf3-40357ddb139e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:01.735424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239da0a0-7c5a-4a52-8cf3-4ccd902f3b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:01.736265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da0a0-7c5a-4a52-8cf3-4ccd902f3b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.736610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da0a0-7c5a-4a52-8cf3-4ccd902f3b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.737212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239da0a0-7c5a-4a52-8cf3-4ccd902f3b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.739151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239da0a0-7c5a-4a52-8cf3-4ccd902f3b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:01.740805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239da0a0-7c5a-4a52-8cf3-4ccd902f3b8a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.753622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 926/1844] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:01.757641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:01.770852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ee76-e0a6-4d07-8f48-52d0897aeb72 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:01.774629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:01.783526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:01.783526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.783526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.783526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] PATH_INFO: `/v3/projects/6ab84777caa047ed8f44fdb93133fd70/users/20d12e0b7811494faa79059c8052d8c0/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.800268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab84777caa047ed8f44fdb93133fd70, user_id=20d12e0b7811494faa79059c8052d8c0, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:01.802982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:01.914552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 919/1845] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:01 2022] PUT /identity/v3/projects/6ab84777caa047ed8f44fdb93133fd70/users/20d12e0b7811494faa79059c8052d8c0/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:01.925483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:01.925901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:01.926044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:01.952409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] MFA Rules not processed for user `20d12e0b7811494faa79059c8052d8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:01.953303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:02.006403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:02.008380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:02.024807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 927/1846] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:02.040289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:02.040289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:02.096297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c235636-77af-4b66-9cd6-6651e4424d39 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:02.099085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b944df-f516-467f-976e-c67cc5b27146 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:02.102968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:02.103847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:02.104137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:02.104420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] PATH_INFO: `/v3/projects/29215561b45a4fc78c7538134857b275` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:02.105735 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=29215561b45a4fc78c7538134857b275)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:02.107735 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:02.107955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:02.109157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:02.109693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:02.110324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:02.124406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:02.125895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:02.138804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 928/1847] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:02.160598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:02.171621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50325-5bee-41dc-9b13-3c03ab9ac250 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:02.176723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:02.177723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:02.178090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:02.178447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] PATH_INFO: `/v3/users/20d12e0b7811494faa79059c8052d8c0` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:02.179858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d12e0b7811494faa79059c8052d8c0)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:02.182143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:02.292013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29215561b45a4fc78c7538134857b275', 'request_id': 'req-72f20ee6-583e-4f13-987b-031b3c1beddc'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.297705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29215561b45a4fc78c7538134857b275', 'request_id': 'req-72f20ee6-583e-4f13-987b-031b3c1beddc'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.303881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29215561b45a4fc78c7538134857b275', 'request_id': 'req-72f20ee6-583e-4f13-987b-031b3c1beddc'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.392495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:02.426297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] The token cache is being invalidate because project 29215561b45a4fc78c7538134857b275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:02.426833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.428314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.430306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 920/1848] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:02 2022] DELETE /identity/v3/projects/29215561b45a4fc78c7538134857b275 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:02.438200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:02.438200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:02.440846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d12e0b7811494faa79059c8052d8c0'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.441609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d12e0b7811494faa79059c8052d8c0'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.442303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d12e0b7811494faa79059c8052d8c0'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.449826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d12e0b7811494faa79059c8052d8c0'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.508237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f20ee6-583e-4f13-987b-031b3c1beddc admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:02.512611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fd3a7df9-b9c2-4733-b217-0c7e6bfe92dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d12e0b7811494faa79059c8052d8c0'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.513415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:02.514403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:02.515373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:02.515686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] PATH_INFO: `/v3/users/45529c5ab4c347e4b0736d7691467ed8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:02.517117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=45529c5ab4c347e4b0736d7691467ed8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:02.519143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:02.541699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 929/1849] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:02 2022] DELETE /identity/v3/users/20d12e0b7811494faa79059c8052d8c0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:02.546055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:02.546351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:02.546637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:02.563666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:02.564995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:02.623255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 930/1850] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:02.633055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:02.634388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:02.690025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d7ee3-1673-413c-9683-d966bf1562c8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:02.693686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:02.694345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:02.694634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:02.694915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:02.704907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:02.707866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:02.711691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45529c5ab4c347e4b0736d7691467ed8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.719279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 931/1851] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:02.724636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45529c5ab4c347e4b0736d7691467ed8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.729284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45529c5ab4c347e4b0736d7691467ed8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.729284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:02.729284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45529c5ab4c347e4b0736d7691467ed8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.736547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45529c5ab4c347e4b0736d7691467ed8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:02.805541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 921/1852] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:02 2022] DELETE /identity/v3/users/45529c5ab4c347e4b0736d7691467ed8 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:03.312457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:03.313980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:03.326791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297611-29c8-4c0b-b5ef-5f76f4608b7c service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:03.396425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:03.399453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:03.400404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:03.400783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:03.401131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] PATH_INFO: `/v3/projects/6ab84777caa047ed8f44fdb93133fd70` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:03.402558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ab84777caa047ed8f44fdb93133fd70)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:03.404787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:03.435937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:03.438326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:03.467894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31839b96-6834-4146-9bac-2d8000756a7f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:03.475915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:03.484290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:03.484632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:03.484920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] PATH_INFO: `/v3/projects/308aaff4a31e4caa892914e2adca45ac` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:03.486652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=308aaff4a31e4caa892914e2adca45ac)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:03.489473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:03.595864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ab84777caa047ed8f44fdb93133fd70', 'request_id': 'req-56779074-2030-419f-9b1c-ede897122e84'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.603891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ab84777caa047ed8f44fdb93133fd70', 'request_id': 'req-56779074-2030-419f-9b1c-ede897122e84'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.619698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '308aaff4a31e4caa892914e2adca45ac', 'request_id': 'req-ebcaaf30-b910-49d2-b978-17ce4f178f29'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.625793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '308aaff4a31e4caa892914e2adca45ac', 'request_id': 'req-ebcaaf30-b910-49d2-b978-17ce4f178f29'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.632831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '308aaff4a31e4caa892914e2adca45ac', 'request_id': 'req-ebcaaf30-b910-49d2-b978-17ce4f178f29'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.702726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ab84777caa047ed8f44fdb93133fd70', 'request_id': 'req-56779074-2030-419f-9b1c-ede897122e84'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.714164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] The token cache is being invalidate because project 6ab84777caa047ed8f44fdb93133fd70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:03.714456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.715121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-56779074-2030-419f-9b1c-ede897122e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.716773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 932/1853] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:03 2022] DELETE /identity/v3/projects/6ab84777caa047ed8f44fdb93133fd70 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:03.722305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:03.722305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:03.722564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:03.742274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:03.743549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:03.812362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] The token cache is being invalidate because project 308aaff4a31e4caa892914e2adca45ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:03.812853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.813960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ebcaaf30-b910-49d2-b978-17ce4f178f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:03.814498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 933/1854] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:03.815814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 922/1855] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:03 2022] DELETE /identity/v3/projects/308aaff4a31e4caa892914e2adca45ac => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:03.819822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:03.821259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:03.823944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26e3cb-43c2-46f0-a0f9-33e654986d6b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:03.825084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26e3cb-43c2-46f0-a0f9-33e654986d6b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:03.825898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26e3cb-43c2-46f0-a0f9-33e654986d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:03.834942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24bb2d-499d-4f7e-ab6d-37c94cf343ad None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:03.840620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:03.841501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:03.841793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:03.842077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] PATH_INFO: `/v3/users/53ea326186a1438d939a6555c7525175` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:03.844843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ea326186a1438d939a6555c7525175)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:03.846830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:03.851007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ea26e3cb-43c2-46f0-a0f9-33e654986d6b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:03.852301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ea26e3cb-43c2-46f0-a0f9-33e654986d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:03.864654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 923/1856] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:03.865840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26e3cb-43c2-46f0-a0f9-33e654986d6b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:03.876092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26e3cb-43c2-46f0-a0f9-33e654986d6b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:03.877907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:03.882199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:03.882866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:03.883152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:03.883431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:03.884167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:03.884558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:03.886230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:03.902784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 924/1857] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:50:03.904166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:03.916843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769bec7-7247-4932-8ef7-8b4b7692b997 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:03.923084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:03.923763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:03.924049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:03.924329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:03.925116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:03.927064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:04.038997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 925/1858] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:03 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:04.041519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:04.104531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1219ee1-325f-47b1-867f-d66eee0b1561 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:04.113390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:04.114183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:04.114748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:04.115197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:04.115975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:04.118166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:04.127491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ea326186a1438d939a6555c7525175'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:04.128737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ea326186a1438d939a6555c7525175'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:04.130147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ea326186a1438d939a6555c7525175'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:04.140232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ea326186a1438d939a6555c7525175'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:04.201875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 926/1859] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:04 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:04.205407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:04.214076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:04.229083 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ea326186a1438d939a6555c7525175'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:04.249920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37679090-20da-4799-a8d6-cf46208b16d2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:04.263260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 934/1860] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:03 2022] DELETE /identity/v3/users/53ea326186a1438d939a6555c7525175 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:04.264305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:04.265130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:04.265514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:04.267346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:04.267346 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:04.268786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:04.279570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 927/1861] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:04.286609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:04.368392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca133d18-ed81-467f-aa61-1b675ebb2595 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:04.374354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb5a14-5dca-4d97-8d2e-983e0fc1b395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:04.378571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5a14-5dca-4d97-8d2e-983e0fc1b395 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:04.378571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5a14-5dca-4d97-8d2e-983e0fc1b395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:04.378571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5a14-5dca-4d97-8d2e-983e0fc1b395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:04.378571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb5a14-5dca-4d97-8d2e-983e0fc1b395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:04.378571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb5a14-5dca-4d97-8d2e-983e0fc1b395 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:04.392032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 935/1862] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:04.396987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:04.409503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a1eef-c76e-4392-9a25-aa2125585cc9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:04.417081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:04.418341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:04.418896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:04.419379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] PATH_INFO: `/v3/projects/64254ff60fcc48a2ba82aa7dafc35709/users/384026a0822a4d7fa04cde1d20c6822b/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:04.436783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64254ff60fcc48a2ba82aa7dafc35709, user_id=384026a0822a4d7fa04cde1d20c6822b, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:04.441042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:04.536600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:04.546384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 928/1863] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:04 2022] PUT /identity/v3/projects/64254ff60fcc48a2ba82aa7dafc35709/users/384026a0822a4d7fa04cde1d20c6822b/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:04.557564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:04.558388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:04.559603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:04.608130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] MFA Rules not processed for user `384026a0822a4d7fa04cde1d20c6822b`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:04.610140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:04.694414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 936/1864] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:04.710139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:04.712054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:04.864649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11fa1a-c9ec-4793-809d-2f0db08bd13d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:04.881345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:04.882057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:04.882380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:04.882579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:04.898100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:04.899572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:04.909592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 929/1865] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:05.006240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:05.230975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:05.232756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:05.313964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1eb6b8-7045-4d80-a7d1-1886419259b1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:05.318997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:05.319866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:05.320148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:05.320426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] PATH_INFO: `/v3/projects/7a61cfabfa0c43eea5fd0f00636de434` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:05.321640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a61cfabfa0c43eea5fd0f00636de434)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:05.323546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:05.430497 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a61cfabfa0c43eea5fd0f00636de434', 'request_id': 'req-34305532-39f8-4804-af8e-a47e8bb7a8e4'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:05.436861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a61cfabfa0c43eea5fd0f00636de434', 'request_id': 'req-34305532-39f8-4804-af8e-a47e8bb7a8e4'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:05.588848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a61cfabfa0c43eea5fd0f00636de434', 'request_id': 'req-34305532-39f8-4804-af8e-a47e8bb7a8e4'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:05.608063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:05.610355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:05.611240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] The token cache is being invalidate because project 7a61cfabfa0c43eea5fd0f00636de434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:05.611706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:05.612536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34305532-39f8-4804-af8e-a47e8bb7a8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:05.614617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 937/1866] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:05 2022] DELETE /identity/v3/projects/7a61cfabfa0c43eea5fd0f00636de434 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:05.624152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266a1f7-6e31-46a2-bbb3-af05d5155e8c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:05.624724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266a1f7-6e31-46a2-bbb3-af05d5155e8c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:05.624857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd69f7-3913-44bb-a38f-036bbca298bb service glance] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:05.625279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266a1f7-6e31-46a2-bbb3-af05d5155e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:05.654644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-2266a1f7-6e31-46a2-bbb3-af05d5155e8c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:05.665992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2266a1f7-6e31-46a2-bbb3-af05d5155e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:05.698679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:05.699959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:05.700351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:05.700874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] PATH_INFO: `/v3/users/349efd8ac7e94760b00be3216c970704` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:05.702462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349efd8ac7e94760b00be3216c970704)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:05.705612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:05.745318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 938/1867] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:05.754713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8344c-2cdf-472f-a0d1-2d9033599b9c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:05.756770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8344c-2cdf-472f-a0d1-2d9033599b9c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:05.757686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8344c-2cdf-472f-a0d1-2d9033599b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:05.778238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b3d8344c-2cdf-472f-a0d1-2d9033599b9c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:05.779450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b3d8344c-2cdf-472f-a0d1-2d9033599b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:05.788383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 939/1868] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:05.792665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8344c-2cdf-472f-a0d1-2d9033599b9c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:05.806093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8344c-2cdf-472f-a0d1-2d9033599b9c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:05.811536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:05.812319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:05.812740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:05.813146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:05.813991 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:05.814406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:05.816107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:05.830738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 940/1869] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:50:05.835007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:05.853227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552df8b-2a19-4817-b587-450731633eb2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:05.859530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:05.860306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:05.860660 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:05.861524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:05.862320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:05.867928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:07.446218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 941/1870] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:05 2022] POST /identity/v3/projects => generated 375 bytes in 1612 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:07.450638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:07.630429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:07.705114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332521de-84d2-493d-a0f7-b571c152c1a1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:07.709742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:07.710470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:07.710764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:07.711013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:07.711711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:07.713678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:07.752662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:07.754749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349efd8ac7e94760b00be3216c970704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:07.767407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349efd8ac7e94760b00be3216c970704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:07.768120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349efd8ac7e94760b00be3216c970704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:07.768781 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349efd8ac7e94760b00be3216c970704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:07.777916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ab042da8-4d0b-4859-83af-104e821109a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349efd8ac7e94760b00be3216c970704'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:07.837325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 942/1871] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:07 2022] POST /identity/v3/users => generated 288 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:07.837571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:07.839187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:07.851148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7ca4b-7a8f-4da7-afc8-bebad9adfa24 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:07.856448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:07.857156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:07.857409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:07.857681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:07.858415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:07.860052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:07.863777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 930/1872] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:05 2022] DELETE /identity/v3/users/349efd8ac7e94760b00be3216c970704 => generated 0 bytes in 2256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:07.869105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:07.869395 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:07.869674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:07.873484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 943/1873] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:07.874775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:07.884210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51c4b-e2b0-4358-b2e4-e1c4edbbcf62 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:07.887132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:07.888325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:07.945840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:07.946460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:07.946751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:07.948595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:07.949006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 931/1874] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:07.949357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:07.951020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:07.959132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:07.973204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:07.973327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 944/1875] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:07.976000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:07.985638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc0d4-17d1-454c-9870-f01b403b114a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:07.990221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:07.991101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:07.991387 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:07.991670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] PATH_INFO: `/v3/projects/8f2e8171cb194fd6a33323d5150c7130/users/0ae64357b1984c8eaf950163f7480051/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.008174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2e8171cb194fd6a33323d5150c7130, user_id=0ae64357b1984c8eaf950163f7480051, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.010980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.017145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27c40-6734-409f-9773-47503d7dfb9d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:08.020963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ab08d-5d33-47b5-ad1f-70cb1d7cfdbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:08.021725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab08d-5d33-47b5-ad1f-70cb1d7cfdbd service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.022074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab08d-5d33-47b5-ad1f-70cb1d7cfdbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.022416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab08d-5d33-47b5-ad1f-70cb1d7cfdbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.031982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ab08d-5d33-47b5-ad1f-70cb1d7cfdbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.033484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ab08d-5d33-47b5-ad1f-70cb1d7cfdbd service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.043111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 932/1876] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:08.086098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 945/1877] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:07 2022] PUT /identity/v3/projects/8f2e8171cb194fd6a33323d5150c7130/users/0ae64357b1984c8eaf950163f7480051/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:08.098182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680ccca-e1e9-4c79-a178-9c5d97ac8938 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.098708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680ccca-e1e9-4c79-a178-9c5d97ac8938 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.098834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680ccca-e1e9-4c79-a178-9c5d97ac8938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.127188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a680ccca-e1e9-4c79-a178-9c5d97ac8938 None None] MFA Rules not processed for user `0ae64357b1984c8eaf950163f7480051`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:08.128418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a680ccca-e1e9-4c79-a178-9c5d97ac8938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:08.191496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 933/1878] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:08.196239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:08.233046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:08.259659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68058b5b-4846-4358-a5ce-9a829d330f17 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:08.265548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:08.266278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.266566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.271137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.271137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.271137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.387763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 946/1879] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:08 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:08.389178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680ccca-e1e9-4c79-a178-9c5d97ac8938 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:08.400635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680ccca-e1e9-4c79-a178-9c5d97ac8938 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:08.461995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:08.463686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.464079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.464485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.465337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.468314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.560147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 934/1880] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:08 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:08.564699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:08.574926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa19768-9dff-44e6-80fc-35c3151cadc2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:08.580253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:08.586802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.586802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.586802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.586802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.586802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.595038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 947/1881] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:08.598997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:08.608959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300d9e-c8cb-4fba-877c-b6657c862018 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:08.613719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:08.614373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.617964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.617964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.617964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.617964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.627522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 935/1882] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:08.631701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:08.648663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65590d60-0f02-4b25-84a6-cb3da2bdaee4 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:08.649745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:08.652503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:08.666504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edb41-7bc9-4671-a84d-854307361981 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:08.671745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:08.672839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.673301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.673733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] PATH_INFO: `/v3/projects/9c55263b75d1457690f9c5e5f04124a8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.675016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380f7e4-0b5d-4418-8928-4d2a39f680a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:08.675490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c55263b75d1457690f9c5e5f04124a8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.675935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380f7e4-0b5d-4418-8928-4d2a39f680a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.676218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380f7e4-0b5d-4418-8928-4d2a39f680a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.676699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380f7e4-0b5d-4418-8928-4d2a39f680a0 admin admin] PATH_INFO: `/v3/projects/e0b537475d4c4c36a5d75d1db8c5c4d1/users/080a89302247440abde04b59732f0ee8/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.677439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.697748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380f7e4-0b5d-4418-8928-4d2a39f680a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b537475d4c4c36a5d75d1db8c5c4d1, user_id=080a89302247440abde04b59732f0ee8, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:08.700648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380f7e4-0b5d-4418-8928-4d2a39f680a0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:08.840893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 948/1883] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:08 2022] PUT /identity/v3/projects/e0b537475d4c4c36a5d75d1db8c5c4d1/users/080a89302247440abde04b59732f0ee8/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:08.861707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:08.862359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:08.862858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:08.877264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c55263b75d1457690f9c5e5f04124a8', 'request_id': 'req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:08.883008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c55263b75d1457690f9c5e5f04124a8', 'request_id': 'req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:08.907185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c55263b75d1457690f9c5e5f04124a8', 'request_id': 'req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:08.941669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] MFA Rules not processed for user `080a89302247440abde04b59732f0ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:08.943207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:08.997061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:08.999006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] The token cache is being invalidate because project 9c55263b75d1457690f9c5e5f04124a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:08.999129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:08.999765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-2feb8782-3cb5-43f6-8c7c-5b0ec24aeedb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:09.005408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 936/1884] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:08 2022] DELETE /identity/v3/projects/9c55263b75d1457690f9c5e5f04124a8 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:09.009086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35863e-27c1-4b36-b9bc-c0b97aa373bb None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.009531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35863e-27c1-4b36-b9bc-c0b97aa373bb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.010306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35863e-27c1-4b36-b9bc-c0b97aa373bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.035829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0f35863e-27c1-4b36-b9bc-c0b97aa373bb None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:09.038239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0f35863e-27c1-4b36-b9bc-c0b97aa373bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:09.052601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 949/1885] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:09.057218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.059107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:09.141283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 937/1886] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:09.150025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ac3b-45db-47f4-b358-ee7aedf4a5f2 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.150441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ac3b-45db-47f4-b358-ee7aedf4a5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.150860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f190b26-6fcf-497b-95f2-eb4e35fda936 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.150986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ac3b-45db-47f4-b358-ee7aedf4a5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.156978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.157712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.158056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.158343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.168693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.170681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.179140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cea5ac3b-45db-47f4-b358-ee7aedf4a5f2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:09.179783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cea5ac3b-45db-47f4-b358-ee7aedf4a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:09.192893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 938/1887] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:09.199506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5ac3b-45db-47f4-b358-ee7aedf4a5f2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.204195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5ac3b-45db-47f4-b358-ee7aedf4a5f2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.209213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.209906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.210212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.210457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.211239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:09.211543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.213192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.226698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 939/1888] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:09.230733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.240824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e27870-faac-4d3d-a5e4-26bbb289dc16 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.245956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.246612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.246893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.247165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.247848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.250439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.294602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 950/1889] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/projects => generated 377 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:09.298766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.363709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957d35-5797-447e-b749-b82a76c08a23 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.368395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.369117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.369437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.369718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.370410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.372410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.384901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 940/1890] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:09.389093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.451861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 951/1891] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:09.456329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.461226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54861668-e0a4-4239-829c-03a2a3921250 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.465780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f6c91-42a4-458f-afef-8a56168715b5 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.465938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.466617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.466907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.467187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.467882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.469838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.470639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.471305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.471627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.471935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.472796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.474336 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.488462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 952/1892] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:09.489980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.500346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709a5ca-6475-4cee-aea2-7935f9d13821 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.505262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.506225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.506536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.506815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] PATH_INFO: `/v3/projects/afc9a86f9e1340328fa4f52fe7626a73/users/a1a53cee6cbf465b9189c1ff93317646/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.522533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc9a86f9e1340328fa4f52fe7626a73, user_id=a1a53cee6cbf465b9189c1ff93317646, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.525243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.555470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 941/1893] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:09.559718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.569383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f1f57-be66-4d1d-aa83-a2f73aa60ffb admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.574506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.576234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.576234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.576234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.576633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.578697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.589495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 942/1894] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:09.593501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.601972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 953/1895] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:09 2022] PUT /identity/v3/projects/afc9a86f9e1340328fa4f52fe7626a73/users/a1a53cee6cbf465b9189c1ff93317646/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:09.606821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4f94-4786-429d-8bf1-1cf63d828a67 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.607748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.686085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.687118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.687386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.687847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.688976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.690385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71efeb0-2c5f-4814-b2d3-118255f15a64 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.691024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.698914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.698914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.698914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.698914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.699929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.699929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.709829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 943/1896] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:09.714872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 954/1897] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:09.715007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.715871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.726239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fc682-fa4d-48f3-b7cb-cee82f9dd356 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.727818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dcc67-e6c9-4031-83f5-d138d206527e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:09.731352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e4734-a484-4d5b-92e5-6da6fa6bc4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.732269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e4734-a484-4d5b-92e5-6da6fa6bc4a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.732594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e4734-a484-4d5b-92e5-6da6fa6bc4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.732875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e4734-a484-4d5b-92e5-6da6fa6bc4a6 admin admin] PATH_INFO: `/v3/projects/e481023311984f5cb3388d7354d47fb6/users/3deceff47f984622bce32990d5d5e599/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.733136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445a76-9f73-4d19-b0e5-9d7120df8b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:09.734550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445a76-9f73-4d19-b0e5-9d7120df8b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.734879 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445a76-9f73-4d19-b0e5-9d7120df8b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.735160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445a76-9f73-4d19-b0e5-9d7120df8b0f admin admin] PATH_INFO: `/v3/projects/afc9a86f9e1340328fa4f52fe7626a73/users/a1a53cee6cbf465b9189c1ff93317646/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.737335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445a76-9f73-4d19-b0e5-9d7120df8b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc9a86f9e1340328fa4f52fe7626a73, user_id=a1a53cee6cbf465b9189c1ff93317646, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.740617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445a76-9f73-4d19-b0e5-9d7120df8b0f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.756361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 955/1898] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:09 2022] PUT /identity/v3/projects/afc9a86f9e1340328fa4f52fe7626a73/users/a1a53cee6cbf465b9189c1ff93317646/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:09.756523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e4734-a484-4d5b-92e5-6da6fa6bc4a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e481023311984f5cb3388d7354d47fb6, user_id=3deceff47f984622bce32990d5d5e599, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:09.759470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e4734-a484-4d5b-92e5-6da6fa6bc4a6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:09.766583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.766940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.767334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.795347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] MFA Rules not processed for user `a1a53cee6cbf465b9189c1ff93317646`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:09.797168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:09.868458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 944/1899] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:09 2022] PUT /identity/v3/projects/e481023311984f5cb3388d7354d47fb6/users/3deceff47f984622bce32990d5d5e599/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:09.873467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:09.873778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:09.874104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:09.903152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] MFA Rules not processed for user `3deceff47f984622bce32990d5d5e599`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:09.904365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:09.928485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 956/1900] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:09.938875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:09.942390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:10.034457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 945/1901] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:10.081777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d67f8f-2364-46f3-b360-5bdf9216cb63 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:10.196512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:10.197326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:10.197676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:10.198680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:10.200042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:10.259621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:10.261456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:10.351293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:10.353002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:10.378017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:10.385048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62068ee-4abf-49b7-82a8-db5c6f8d1a4e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:10.388492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 957/1902] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:10.397522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff76f8-53d9-4a1a-9509-d69f9114ef9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:10.399357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff76f8-53d9-4a1a-9509-d69f9114ef9b service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:10.399357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff76f8-53d9-4a1a-9509-d69f9114ef9b service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:10.399357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff76f8-53d9-4a1a-9509-d69f9114ef9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:10.413646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff76f8-53d9-4a1a-9509-d69f9114ef9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:10.416143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff76f8-53d9-4a1a-9509-d69f9114ef9b service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:10.429089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:10.429609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 946/1903] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:10.713391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:10.778336 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905fbce-75c6-422b-b69a-fd59515c3b27 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:10.783635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:10.785011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:10.785586 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:10.785931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] PATH_INFO: `/v3/users/3deceff47f984622bce32990d5d5e599` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:10.787527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3deceff47f984622bce32990d5d5e599)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:10.789516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:10.987485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3deceff47f984622bce32990d5d5e599'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:10.987485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3deceff47f984622bce32990d5d5e599'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:10.987485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3deceff47f984622bce32990d5d5e599'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:10.994889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3deceff47f984622bce32990d5d5e599'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:11.097258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3deceff47f984622bce32990d5d5e599'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:11.116775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 958/1904] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:10 2022] DELETE /identity/v3/users/3deceff47f984622bce32990d5d5e599 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:11.121502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:11.122074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:11.122074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:11.139226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:11.140971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:11.207121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 947/1905] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:11.217017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:11.219014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:11.276279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df603d9-efe4-41d9-8f96-094b33c0b045 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:11.280840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025060-ec11-49a5-92d7-8c58f324cdca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:11.281558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025060-ec11-49a5-92d7-8c58f324cdca service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:11.281850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025060-ec11-49a5-92d7-8c58f324cdca service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:11.282414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025060-ec11-49a5-92d7-8c58f324cdca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:11.292747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37025060-ec11-49a5-92d7-8c58f324cdca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:11.294322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37025060-ec11-49a5-92d7-8c58f324cdca service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:11.310804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 959/1906] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:12.273218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:12.273218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:12.279044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d4624-d847-43c6-98d4-89868f7a904a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:12.284222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:12.285557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.285915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.286265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] PATH_INFO: `/v3/projects/e481023311984f5cb3388d7354d47fb6` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.287471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e481023311984f5cb3388d7354d47fb6)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:12.289718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:12.308020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74970c51-a8d8-4e26-af78-c50486fd867e None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.308914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74970c51-a8d8-4e26-af78-c50486fd867e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.309295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74970c51-a8d8-4e26-af78-c50486fd867e None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.311082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 960/1907] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 17:50:12.331383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.332015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.332304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.352056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:12.353532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:12.429628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 961/1908] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:12.457484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:12.458914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:12.472680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68c474-d091-44f1-91ba-c64741b28e9a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:12.478733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e481023311984f5cb3388d7354d47fb6', 'request_id': 'req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:12.486876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e481023311984f5cb3388d7354d47fb6', 'request_id': 'req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:12.494374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e481023311984f5cb3388d7354d47fb6', 'request_id': 'req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:12.543504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:12.544280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.544618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.544890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.556940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:12.559312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:12.572351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 962/1909] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:12.598579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:12.600278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:12.620030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] The token cache is being invalidate because project e481023311984f5cb3388d7354d47fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:12.620565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:12.621609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-ffa60a2d-e4e8-4d14-a13e-01f1e8bd771d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:12.624641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 948/1910] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:12 2022] DELETE /identity/v3/projects/e481023311984f5cb3388d7354d47fb6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:12.629924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19575207-25d9-4672-a6c7-d0659c1f1507 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.630461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19575207-25d9-4672-a6c7-d0659c1f1507 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.630845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19575207-25d9-4672-a6c7-d0659c1f1507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.674184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:12.674323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3791b28-9357-4bca-bdbe-af1fb2e77440 service nova] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:12.677952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:12.678121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-19575207-25d9-4672-a6c7-d0659c1f1507 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:12.678609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.678897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.679180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.679294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-19575207-25d9-4672-a6c7-d0659c1f1507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:12.681105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:12.776073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:12.778144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:12.783294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 949/1911] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:12.793916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9854b-9538-40f2-9d9d-83a800562960 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.794494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9854b-9538-40f2-9d9d-83a800562960 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.794982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9854b-9538-40f2-9d9d-83a800562960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.797393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 963/1912] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:12.837935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-88d9854b-9538-40f2-9d9d-83a800562960 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:12.839689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-88d9854b-9538-40f2-9d9d-83a800562960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:12.850102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 950/1913] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:12.854154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:12.882145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ff7e-0992-48c6-8df4-c9fb4b3f1733 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:12.882145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:12.882145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.882145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.882532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.882532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:12.882532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:12.882532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:12.902183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 964/1914] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:12.902345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9854b-9538-40f2-9d9d-83a800562960 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:12.913444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9854b-9538-40f2-9d9d-83a800562960 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:12.919066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:12.919746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:12.920079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:12.920362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:12.921901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:12.923934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.046268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 951/1915] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:12 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:13.050362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.061950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24350b-1535-4289-a8c2-16b96c540aa8 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.124821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.125583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.125869 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.126148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.126828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:13.128777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.228663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 965/1916] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:13 2022] POST /identity/v3/users => generated 282 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:13.232841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.243458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d7689-8f29-4966-bac3-db73c9bcd861 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.247982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.248688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.249682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.249963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.250691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:13.252278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.262770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 952/1917] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:13.267450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.279136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55446f3c-3715-42c3-ac1e-8782d07c1613 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.283696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ace02-b6c0-467c-b763-4f3ce0d443f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.284362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ace02-b6c0-467c-b763-4f3ce0d443f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.284686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ace02-b6c0-467c-b763-4f3ce0d443f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.284963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ace02-b6c0-467c-b763-4f3ce0d443f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.285685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ace02-b6c0-467c-b763-4f3ce0d443f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:13.287298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ace02-b6c0-467c-b763-4f3ce0d443f2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.297566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 966/1918] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:13.302083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.313131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495609d2-e52b-430b-85f9-80f37e26b3ca admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.317644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.318590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.318872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.319145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] PATH_INFO: `/v3/projects/7716677f9b4648c2a74f62ce3222cce7/users/1d8f9c57a30c4ffaaa4469feb7dbf37a/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.333175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7716677f9b4648c2a74f62ce3222cce7, user_id=1d8f9c57a30c4ffaaa4469feb7dbf37a, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:13.335951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.451230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 953/1919] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:13 2022] PUT /identity/v3/projects/7716677f9b4648c2a74f62ce3222cce7/users/1d8f9c57a30c4ffaaa4469feb7dbf37a/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:13.461794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee078aa-7d80-438e-8c6b-c6b76fdf51bb None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.462269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee078aa-7d80-438e-8c6b-c6b76fdf51bb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.462382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee078aa-7d80-438e-8c6b-c6b76fdf51bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.492876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4ee078aa-7d80-438e-8c6b-c6b76fdf51bb None None] MFA Rules not processed for user `1d8f9c57a30c4ffaaa4469feb7dbf37a`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:13.503918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4ee078aa-7d80-438e-8c6b-c6b76fdf51bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:13.579861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 967/1920] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:13.581571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.642500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5b2b8-e416-44a4-afd8-02a03299c1b0 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.647205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.647867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.648156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.648459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.649172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:13.651134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.703294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:13.765046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 954/1921] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:13 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:13.769362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee078aa-7d80-438e-8c6b-c6b76fdf51bb None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.780568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee078aa-7d80-438e-8c6b-c6b76fdf51bb None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.834491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.835162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.835453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.835743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.836507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:13.838432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.925249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 968/1922] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:13 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:13.928665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.940226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b9b3-b1f8-46f6-8ffc-8acd2b0b072f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.945752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.946428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.946715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.946983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.947719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:13.949350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:13.961402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 955/1923] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:13.965529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:13.976950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d19ea2-ab4a-42d4-bbba-74767cc6fe24 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:13.981743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:13.982634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:13.982919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:13.983196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] PATH_INFO: `/v3/projects/e1cb2fa9479c4651a20586958565af6b/users/cef97047f789411593df82c343068d4d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:13.998172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1cb2fa9479c4651a20586958565af6b, user_id=cef97047f789411593df82c343068d4d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:14.001674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:14.342646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 969/1924] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:13 2022] PUT /identity/v3/projects/e1cb2fa9479c4651a20586958565af6b/users/cef97047f789411593df82c343068d4d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:14.346694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:14.356598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bc255-430e-4524-84e2-c40e76cab077 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:14.387868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:14.414135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e95458-5b6e-426e-95f5-3ac64e3b81b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:14.414817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95458-5b6e-426e-95f5-3ac64e3b81b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:14.415098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95458-5b6e-426e-95f5-3ac64e3b81b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:14.415371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95458-5b6e-426e-95f5-3ac64e3b81b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:14.416128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e95458-5b6e-426e-95f5-3ac64e3b81b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:14.417756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e95458-5b6e-426e-95f5-3ac64e3b81b6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:14.428395 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 956/1925] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:14.434208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:14.446411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b6f75-4d5c-4981-bed9-7f988788eb3d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:14.458824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:14.458824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:14.458824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:14.458824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] PATH_INFO: `/v3/projects/e1cb2fa9479c4651a20586958565af6b/users/cef97047f789411593df82c343068d4d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:14.459199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1cb2fa9479c4651a20586958565af6b, user_id=cef97047f789411593df82c343068d4d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:14.459199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:14.467389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 970/1926] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:14 2022] PUT /identity/v3/projects/e1cb2fa9479c4651a20586958565af6b/users/cef97047f789411593df82c343068d4d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:14.477937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48dd9-e7f1-41d2-92f2-82270f9eaac0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:14.478225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48dd9-e7f1-41d2-92f2-82270f9eaac0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:14.478529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48dd9-e7f1-41d2-92f2-82270f9eaac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:14.504935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8bb48dd9-e7f1-41d2-92f2-82270f9eaac0 None None] MFA Rules not processed for user `cef97047f789411593df82c343068d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:14.505719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8bb48dd9-e7f1-41d2-92f2-82270f9eaac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:14.578986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 957/1927] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:14.590356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:14.592048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:14.649881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78392-3bc1-44c5-983e-658d7896481c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:14.654514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:14.656022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:14.656399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:14.656709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:14.666308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:14.667743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:14.677893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 971/1928] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:14.898279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb48dd9-e7f1-41d2-92f2-82270f9eaac0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:14.908179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb48dd9-e7f1-41d2-92f2-82270f9eaac0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:14.911686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:14.912320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:14.912698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:14.912977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:14.968627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:14.970122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:14.978702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 958/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:15.537310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:15.920560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:16.277763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:16.287020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff2658-17ae-4106-a936-c57317342c10 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:16.290988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:16.291635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:16.291981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:16.292268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:16.294067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:16.377844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:16.377844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:16.388752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 972/1930] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:16.398571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:17.631792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:17.708720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:19.101711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:19.166338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:20.264132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:20.440579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:21.137732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:21.245513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:21.560260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:21.628904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:21.903027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:21.931436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:22.074348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:22.119800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:22.268487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:22.297413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:22.354019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:22.866855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:22.994191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:23.051258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:23.211232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:23.243873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:23.342645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:23.371194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:25.530011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:25.531734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:25.656852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd038-3712-44d8-83ab-d4abfa07ebe7 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:25.672090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:25.673135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:25.673348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:25.673622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] PATH_INFO: `/v3/users/0ae64357b1984c8eaf950163f7480051` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:25.678582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae64357b1984c8eaf950163f7480051)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:25.683070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:25.968718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:25.971532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:25.997611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed94d7c-88df-4839-9f75-7546bdea0cac service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:26.011833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:26.013380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:26.014128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:26.015391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] PATH_INFO: `/v3/users/e829024f3bba48119b610328a92034a9` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:26.018022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e829024f3bba48119b610328a92034a9)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:26.021952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:26.028417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae64357b1984c8eaf950163f7480051'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.053093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae64357b1984c8eaf950163f7480051'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.053872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae64357b1984c8eaf950163f7480051'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.054596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae64357b1984c8eaf950163f7480051'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.065145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-65f81251-afff-4591-b4ab-d4942ff04d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae64357b1984c8eaf950163f7480051'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.126757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 959/1931] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:25 2022] DELETE /identity/v3/users/0ae64357b1984c8eaf950163f7480051 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:26.131077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:26.131627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:26.132153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:26.154309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:26.156628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:26.212462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:26.232026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 960/1932] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:26.244310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:26.245970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:26.249835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e829024f3bba48119b610328a92034a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.250516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e829024f3bba48119b610328a92034a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.251240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e829024f3bba48119b610328a92034a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.259237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e829024f3bba48119b610328a92034a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.305502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffad8c9-6586-4c36-8c8c-19a30245af58 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:26.309472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3a381-a185-4c2f-bb59-c8ff2fe08f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:26.310157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3a381-a185-4c2f-bb59-c8ff2fe08f16 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:26.310445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3a381-a185-4c2f-bb59-c8ff2fe08f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:26.310726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3a381-a185-4c2f-bb59-c8ff2fe08f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:26.313228 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e8f3a381-a185-4c2f-bb59-c8ff2fe08f16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:26.324774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3a381-a185-4c2f-bb59-c8ff2fe08f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:26.326201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3a381-a185-4c2f-bb59-c8ff2fe08f16 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:26.337417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 961/1933] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:26.342901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e829024f3bba48119b610328a92034a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:26.358702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:26.363610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 973/1934] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:25 2022] DELETE /identity/v3/users/e829024f3bba48119b610328a92034a9 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:26.368037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:26.368331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:26.368956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:26.393701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:26.395479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:26.467413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 962/1935] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:26.477747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:26.479239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:26.556105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33157-de9b-4b01-a849-09a1526b931a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:26.560860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:26.562813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:26.562813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:26.562813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:26.575271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:26.579340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:26.589530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 974/1936] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:26.926605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:26.928178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:26.943155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae47f2-bf91-461d-95bd-47c66c0963e5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:26.949163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:26.952297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:26.952297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:26.952297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] PATH_INFO: `/v3/projects/8f2e8171cb194fd6a33323d5150c7130` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:26.952765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2e8171cb194fd6a33323d5150c7130)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:26.954877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:27.082241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2e8171cb194fd6a33323d5150c7130', 'request_id': 'req-c8a80344-cf30-40a1-99b9-1c7fc55d4428'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.090562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2e8171cb194fd6a33323d5150c7130', 'request_id': 'req-c8a80344-cf30-40a1-99b9-1c7fc55d4428'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.115085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2e8171cb194fd6a33323d5150c7130', 'request_id': 'req-c8a80344-cf30-40a1-99b9-1c7fc55d4428'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.182161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:27.185341 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:27.212348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] The token cache is being invalidate because project 8f2e8171cb194fd6a33323d5150c7130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:27.214144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.216772 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.230381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 963/1937] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:26 2022] DELETE /identity/v3/projects/8f2e8171cb194fd6a33323d5150c7130 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:27.232956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:27.234766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1700-40f1-4553-b121-192e1673f3e0 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:27.238253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:27.273353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:27.364462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:27.365725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:27.367093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:27.367369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a80344-cf30-40a1-99b9-1c7fc55d4428 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:27.367818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] PATH_INFO: `/v3/projects/f768f28d1cdc41d1afbd15edcdf14268` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:27.370140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f768f28d1cdc41d1afbd15edcdf14268)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:27.372910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:27.374335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:27.374618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:27.374906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] PATH_INFO: `/v3/users/080a89302247440abde04b59732f0ee8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:27.376541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080a89302247440abde04b59732f0ee8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:27.377549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:27.378885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:27.579479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080a89302247440abde04b59732f0ee8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.591951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f768f28d1cdc41d1afbd15edcdf14268', 'request_id': 'req-08f41cd0-b9e4-4356-a1af-1a7fc53df677'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.609394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f768f28d1cdc41d1afbd15edcdf14268', 'request_id': 'req-08f41cd0-b9e4-4356-a1af-1a7fc53df677'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.619877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080a89302247440abde04b59732f0ee8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.621717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080a89302247440abde04b59732f0ee8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.621717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080a89302247440abde04b59732f0ee8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.631461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080a89302247440abde04b59732f0ee8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.767980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f768f28d1cdc41d1afbd15edcdf14268', 'request_id': 'req-08f41cd0-b9e4-4356-a1af-1a7fc53df677'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.781864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] The token cache is being invalidate because project f768f28d1cdc41d1afbd15edcdf14268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:27.783253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.784105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:27.787217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 975/1938] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:27 2022] DELETE /identity/v3/projects/f768f28d1cdc41d1afbd15edcdf14268 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:27.794045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:27.797385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:27.857817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:27.898074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f41cd0-b9e4-4356-a1af-1a7fc53df677 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:27.903306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 964/1939] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:27 2022] DELETE /identity/v3/users/080a89302247440abde04b59732f0ee8 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:27.909166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:27.910278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:27.910704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:27.911158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] PATH_INFO: `/v3/users/91a2584742c649cbae59b6684e631e77` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:27.912602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a2584742c649cbae59b6684e631e77)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:27.914773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:28.129845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a2584742c649cbae59b6684e631e77'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.130621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a2584742c649cbae59b6684e631e77'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.132284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a2584742c649cbae59b6684e631e77'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.142680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a2584742c649cbae59b6684e631e77'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.257541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a2584742c649cbae59b6684e631e77'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.283881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 976/1940] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:27 2022] DELETE /identity/v3/users/91a2584742c649cbae59b6684e631e77 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:28.436843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:28.438412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:28.537866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e17fe-ecbd-43d4-b5cd-77cbb03b0999 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:28.569855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:28.594320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:28.595430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:28.595716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:28.595995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] PATH_INFO: `/v3/projects/e0b537475d4c4c36a5d75d1db8c5c4d1` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:28.597371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b537475d4c4c36a5d75d1db8c5c4d1)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:28.599303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:28.749988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b537475d4c4c36a5d75d1db8c5c4d1', 'request_id': 'req-065462e4-25e8-40da-a024-10236007f698'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.756336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b537475d4c4c36a5d75d1db8c5c4d1', 'request_id': 'req-065462e4-25e8-40da-a024-10236007f698'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.763406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b537475d4c4c36a5d75d1db8c5c4d1', 'request_id': 'req-065462e4-25e8-40da-a024-10236007f698'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.883984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] The token cache is being invalidate because project e0b537475d4c4c36a5d75d1db8c5c4d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:28.884231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.888009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:28.889605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:28.891626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:28.898821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 965/1941] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:28 2022] DELETE /identity/v3/projects/e0b537475d4c4c36a5d75d1db8c5c4d1 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:28.908565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:28.909168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3fc6-4512-4440-8c21-0a39b3743e09 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:28.910181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.001041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:29.002203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.003226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.003661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] PATH_INFO: `/v3/projects/ff857e68441944dda1e34365118a7fb9` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.005510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff857e68441944dda1e34365118a7fb9)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:29.008169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:29.010566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065462e4-25e8-40da-a024-10236007f698 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:29.016311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:29.018449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.018771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.019050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] PATH_INFO: `/v3/users/a1a53cee6cbf465b9189c1ff93317646` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.020706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a53cee6cbf465b9189c1ff93317646)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:29.027732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:29.161164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff857e68441944dda1e34365118a7fb9', 'request_id': 'req-d3faca2d-a2c5-4694-8dc7-60464452c099'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.167914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff857e68441944dda1e34365118a7fb9', 'request_id': 'req-d3faca2d-a2c5-4694-8dc7-60464452c099'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.170452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:29.245914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff857e68441944dda1e34365118a7fb9', 'request_id': 'req-d3faca2d-a2c5-4694-8dc7-60464452c099'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.262416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] The token cache is being invalidate because project ff857e68441944dda1e34365118a7fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:29.263001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.263417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.265581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 977/1942] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:28 2022] DELETE /identity/v3/projects/ff857e68441944dda1e34365118a7fb9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:29.267037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a53cee6cbf465b9189c1ff93317646'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.269820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:29.271318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.287977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a53cee6cbf465b9189c1ff93317646'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.289900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a53cee6cbf465b9189c1ff93317646'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.290120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a53cee6cbf465b9189c1ff93317646'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.299327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-11043f3c-151b-4a07-a606-9fee4791e8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a53cee6cbf465b9189c1ff93317646'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.344348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faca2d-a2c5-4694-8dc7-60464452c099 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:29.349025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:29.349914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.350298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.350479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] PATH_INFO: `/v3/users/c8d90ec4f0c240edba386f248ec8906c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.352009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d90ec4f0c240edba386f248ec8906c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:29.354410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:29.383354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 966/1943] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:28 2022] DELETE /identity/v3/users/a1a53cee6cbf465b9189c1ff93317646 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:29.384764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.385117 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.385489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.408839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:29.410060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.440263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:29.479189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 967/1944] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:29.486726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:29.487924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.500216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9daf88-08a6-4b55-b65c-f847dba0586f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:29.504941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:29.505814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.506098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.506373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] PATH_INFO: `/v3/users/1d8f9c57a30c4ffaaa4469feb7dbf37a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.507568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d8f9c57a30c4ffaaa4469feb7dbf37a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:29.509537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:29.527031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d90ec4f0c240edba386f248ec8906c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.527742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d90ec4f0c240edba386f248ec8906c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.528375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d90ec4f0c240edba386f248ec8906c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.535315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d90ec4f0c240edba386f248ec8906c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.592943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-415c50c7-1913-471d-8e4a-fe38ccf18fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d90ec4f0c240edba386f248ec8906c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.614076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 978/1945] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:29 2022] DELETE /identity/v3/users/c8d90ec4f0c240edba386f248ec8906c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:29.619418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547b3f-83af-4f72-9702-0df9d18e9fe6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.619418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547b3f-83af-4f72-9702-0df9d18e9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.619418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547b3f-83af-4f72-9702-0df9d18e9fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.636882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-04547b3f-83af-4f72-9702-0df9d18e9fe6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:29.638215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-04547b3f-83af-4f72-9702-0df9d18e9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.702912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d8f9c57a30c4ffaaa4469feb7dbf37a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.705218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 979/1946] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:29.707099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.707407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.708714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.724592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d8f9c57a30c4ffaaa4469feb7dbf37a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.725953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d8f9c57a30c4ffaaa4469feb7dbf37a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.725953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d8f9c57a30c4ffaaa4469feb7dbf37a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.733426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:29.734753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.735065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d8f9c57a30c4ffaaa4469feb7dbf37a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:29.743290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:29.745278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 980/1947] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:29.746636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:29.748023 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.767127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9372a5-f0bb-4971-b14b-6d3a3d35dc1e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:29.771834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:29.772533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.772823 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.773101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.773829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:29.774138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:29.775732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:29.787601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 981/1948] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:29.788401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:29.789978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.836464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 968/1949] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:29 2022] DELETE /identity/v3/users/1d8f9c57a30c4ffaaa4469feb7dbf37a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:29.836711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:29.854157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eea70-9e05-45d2-82d8-789110059e94 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:29.925914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158b5a4-a509-4123-a3db-d7a8d201cfcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:29.925914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158b5a4-a509-4123-a3db-d7a8d201cfcd service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.925914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158b5a4-a509-4123-a3db-d7a8d201cfcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.925914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158b5a4-a509-4123-a3db-d7a8d201cfcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.932462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fb772-c2b0-458c-8c49-c9abab856894 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:29.937663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158b5a4-a509-4123-a3db-d7a8d201cfcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:29.937997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:29.938663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.938953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.939223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.939719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158b5a4-a509-4123-a3db-d7a8d201cfcd service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:29.939910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:29.941851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:29.949236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:29.963541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 982/1950] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:29.963541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:29.963541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:29.963541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:29.983668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:29.984927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:29.992969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 983/1951] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:29.993975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:29.995407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:30.007679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838e8fd-1ee3-496d-b137-2e2311a5c726 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.012666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.013538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.013824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.014136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] PATH_INFO: `/v3/projects/afc9a86f9e1340328fa4f52fe7626a73` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.015469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc9a86f9e1340328fa4f52fe7626a73)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.017454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.053599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 969/1952] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:29 2022] POST /identity/v3/projects => generated 407 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:30.054445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.111137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f1bd1-ea63-4266-9131-ea70a1fe3435 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.114861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.115524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.118212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.118212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.154625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc9a86f9e1340328fa4f52fe7626a73', 'request_id': 'req-3ea838b5-3650-4c7a-8d33-e87935b8ead5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.163314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc9a86f9e1340328fa4f52fe7626a73', 'request_id': 'req-3ea838b5-3650-4c7a-8d33-e87935b8ead5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.177945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.179373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.189879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 970/1953] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:30.191516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.199475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fa9dc-43c2-4a44-b863-40b364497761 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.234938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc9a86f9e1340328fa4f52fe7626a73', 'request_id': 'req-3ea838b5-3650-4c7a-8d33-e87935b8ead5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.250819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] The token cache is being invalidate because project afc9a86f9e1340328fa4f52fe7626a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:30.251617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.252796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3ea838b5-3650-4c7a-8d33-e87935b8ead5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.256050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 984/1954] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:29 2022] DELETE /identity/v3/projects/afc9a86f9e1340328fa4f52fe7626a73 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:30.259392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.260549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.260856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.261148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.262471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.264074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723eef0c-d20c-4391-a3b5-d539411e8024 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.266165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723eef0c-d20c-4391-a3b5-d539411e8024 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.266165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723eef0c-d20c-4391-a3b5-d539411e8024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.266558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.287274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-723eef0c-d20c-4391-a3b5-d539411e8024 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:30.289086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-723eef0c-d20c-4391-a3b5-d539411e8024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:30.299297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 985/1955] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:30.308176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1848bb-a849-4d58-a0e9-3b542e63ee95 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.308768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1848bb-a849-4d58-a0e9-3b542e63ee95 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.309051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1848bb-a849-4d58-a0e9-3b542e63ee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.333365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ef1848bb-a849-4d58-a0e9-3b542e63ee95 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:30.334662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ef1848bb-a849-4d58-a0e9-3b542e63ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:30.343441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 986/1956] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:30.347728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1848bb-a849-4d58-a0e9-3b542e63ee95 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.353288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:30.357220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1848bb-a849-4d58-a0e9-3b542e63ee95 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.363167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.363836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.364128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.364412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.365182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:30.365494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.367085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.378742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 987/1957] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:50:30.383046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.390410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 971/1958] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:30 2022] POST /identity/v3/users => generated 304 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:30.393223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a1d9-a40a-4532-bf5c-60fae6e89a1a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.394775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.396407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:30.397991 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.398651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.398934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.399209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.399936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.401938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.411585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d9a47-b013-40ac-9f13-4a5535881b17 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.416908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.417796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.418229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.418832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.421424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.421596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.432131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 972/1959] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:30.436271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.446014 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508af49d-f533-4cc2-a1cb-7f3b2d0e5c27 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.451201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.451926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.452219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.452541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.453288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.454946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.465258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 973/1960] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:30.470557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.475836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:30.482882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad45ea2-3a85-469b-9cd4-e15f6fba08f9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.488562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.489699 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.489990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.490270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] PATH_INFO: `/v3/projects/d2cb127338f440628d8c250c16c1c047/users/8bbbcc05646b4f4e9a88e513b9ff0f69/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.505612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2cb127338f440628d8c250c16c1c047, user_id=8bbbcc05646b4f4e9a88e513b9ff0f69, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.508529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.528022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 988/1961] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:30 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:30.529317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.531675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:30.595281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 974/1962] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:30 2022] PUT /identity/v3/projects/d2cb127338f440628d8c250c16c1c047/users/8bbbcc05646b4f4e9a88e513b9ff0f69/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:30.602071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7426f-8f77-49bc-ab02-626753cbbe03 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.602410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.657544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.658512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.658862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.659216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] PATH_INFO: `/v3/projects/ec32a5930db347bea1ec76fa7e25794c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.660386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec32a5930db347bea1ec76fa7e25794c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.662315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.668193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e4962-235f-43e0-b066-4af920cd6cca admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.673085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123403b-55c8-49da-9902-133b737a3377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.673764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123403b-55c8-49da-9902-133b737a3377 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.674055 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123403b-55c8-49da-9902-133b737a3377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.674335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123403b-55c8-49da-9902-133b737a3377 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.675391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123403b-55c8-49da-9902-133b737a3377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.676986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123403b-55c8-49da-9902-133b737a3377 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.789189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 975/1963] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:30 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:30.793548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b878b-3ef2-4a97-9d3d-661dda9b9103 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.793882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b878b-3ef2-4a97-9d3d-661dda9b9103 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.794227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b878b-3ef2-4a97-9d3d-661dda9b9103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.800766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec32a5930db347bea1ec76fa7e25794c', 'request_id': 'req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.807130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec32a5930db347bea1ec76fa7e25794c', 'request_id': 'req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.820090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-892b878b-3ef2-4a97-9d3d-661dda9b9103 None None] MFA Rules not processed for user `8bbbcc05646b4f4e9a88e513b9ff0f69`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:30.821339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-892b878b-3ef2-4a97-9d3d-661dda9b9103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:30.862631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec32a5930db347bea1ec76fa7e25794c', 'request_id': 'req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.873093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] The token cache is being invalidate because project ec32a5930db347bea1ec76fa7e25794c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:30.873399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.874047 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:30.875601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 989/1964] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:30 2022] DELETE /identity/v3/projects/ec32a5930db347bea1ec76fa7e25794c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:30.876447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.877926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:30.885487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 976/1965] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:30.886410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b878b-3ef2-4a97-9d3d-661dda9b9103 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.951334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaef64e-d7f3-470f-98df-a9f86c3df1e4 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.956964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.960272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:30.960397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.960397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.960397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] PATH_INFO: `/v3/projects/7716677f9b4648c2a74f62ce3222cce7` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.960397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7716677f9b4648c2a74f62ce3222cce7)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.961617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.961855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b878b-3ef2-4a97-9d3d-661dda9b9103 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:30.966719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:30.967980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:30.967980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:30.968128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:30.968997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:30.970775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:30.984859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 977/1966] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:30.984859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:30.988632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.003399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275418ce-a4be-4e47-a202-9bc635cacc7c admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:31.005696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:31.008679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:31.009607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.009888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.010772 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] PATH_INFO: `/v3/users/27adce2541684ca08aa466d1c2a9497b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.011423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27adce2541684ca08aa466d1c2a9497b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:31.013439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:31.075988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7716677f9b4648c2a74f62ce3222cce7', 'request_id': 'req-9f6bacee-370c-4598-9dc8-33481d0c4f8d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.082742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7716677f9b4648c2a74f62ce3222cce7', 'request_id': 'req-9f6bacee-370c-4598-9dc8-33481d0c4f8d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.154200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7716677f9b4648c2a74f62ce3222cce7', 'request_id': 'req-9f6bacee-370c-4598-9dc8-33481d0c4f8d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.165176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] The token cache is being invalidate because project 7716677f9b4648c2a74f62ce3222cce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:31.165468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.166144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.167741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 990/1967] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:30 2022] DELETE /identity/v3/projects/7716677f9b4648c2a74f62ce3222cce7 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:31.168515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:31.169831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.214354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27adce2541684ca08aa466d1c2a9497b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.226190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27adce2541684ca08aa466d1c2a9497b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.226872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27adce2541684ca08aa466d1c2a9497b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.227491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27adce2541684ca08aa466d1c2a9497b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.234487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27adce2541684ca08aa466d1c2a9497b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.235092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bacee-370c-4598-9dc8-33481d0c4f8d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:31.242324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:31.242324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.242324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.242324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.242901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:31.243474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:31.352127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 978/1968] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:30 2022] DELETE /identity/v3/users/27adce2541684ca08aa466d1c2a9497b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:31.353134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:31.354453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.385923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 991/1969] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:31 2022] POST /identity/v3/projects => generated 405 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:31.387565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:31.390753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.426372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91045fcd-9f43-4eff-ba8d-6c1367218ec4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:31.470802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250939-4577-4a5e-a378-5e2d41737d56 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:31.475706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:31.476693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.482395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.482395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] PATH_INFO: `/v3/users/cef97047f789411593df82c343068d4d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.482395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef97047f789411593df82c343068d4d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:31.482395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:31.488455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:31.489128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.489410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.489685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.490415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:31.491994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:31.502418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 979/1970] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:31.503242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:31.504925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.518028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd762a-aedc-4e0f-b5f9-5dfcd5a0319b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:31.522519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:31.523188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.523474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.523754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.524538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:31.526433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:31.534433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:31.615774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 980/1971] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:31 2022] POST /identity/v3/users => generated 309 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:31.616578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:31.627143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c320d4-764b-4335-abbf-a1b91f3151a5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:31.632102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:31.633135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.633421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.633727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] PATH_INFO: `/v3/projects/fc7ae0930e6d4affb6c456846594e513/users/642f415de5c04294a8094a3420ba8523/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.650150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7ae0930e6d4affb6c456846594e513, user_id=642f415de5c04294a8094a3420ba8523, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:31.653005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:31.727895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef97047f789411593df82c343068d4d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.728693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef97047f789411593df82c343068d4d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.729371 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef97047f789411593df82c343068d4d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.736685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef97047f789411593df82c343068d4d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.799936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:31.805791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 981/1972] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:31 2022] PUT /identity/v3/projects/fc7ae0930e6d4affb6c456846594e513/users/642f415de5c04294a8094a3420ba8523/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:31.806644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:31.808593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.869591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef2a5e-75f2-4cc3-af19-907def736f44 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:31.874796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffe6e6-feaa-4e94-ad89-c4d010d07bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:31.875461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffe6e6-feaa-4e94-ad89-c4d010d07bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.875751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffe6e6-feaa-4e94-ad89-c4d010d07bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.876033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffe6e6-feaa-4e94-ad89-c4d010d07bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.876791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffe6e6-feaa-4e94-ad89-c4d010d07bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:31.878413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffe6e6-feaa-4e94-ad89-c4d010d07bd6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:31.892737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 982/1973] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:31.892737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0629638-c4d5-4d13-bdfe-dd59cfe22520 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:31.892737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0629638-c4d5-4d13-bdfe-dd59cfe22520 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:31.892737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0629638-c4d5-4d13-bdfe-dd59cfe22520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:31.914374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef97047f789411593df82c343068d4d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:31.918311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a0629638-c4d5-4d13-bdfe-dd59cfe22520 None None] MFA Rules not processed for user `642f415de5c04294a8094a3420ba8523`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:31.919561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a0629638-c4d5-4d13-bdfe-dd59cfe22520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.927247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 992/1974] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:31 2022] DELETE /identity/v3/users/cef97047f789411593df82c343068d4d => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:31.928072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:31.929456 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:31.996513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 983/1975] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:32.000208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0629638-c4d5-4d13-bdfe-dd59cfe22520 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.015933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576caa59-3a31-4815-8e58-981eb6682071 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.016102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0629638-c4d5-4d13-bdfe-dd59cfe22520 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.025198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.025816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.026271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.028714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.028846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.028954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] PATH_INFO: `/v3/projects/8fd8eac40af54075afccaf924f4a87c8/users/a66a3ba723434c589dfd63e3d5fc11e8/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.029043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.029043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] PATH_INFO: `/v3/projects/5f1be7a73dc341c98eed3bac9ccd18ba` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.029043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f1be7a73dc341c98eed3bac9ccd18ba)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.031131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.046359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd8eac40af54075afccaf924f4a87c8, user_id=a66a3ba723434c589dfd63e3d5fc11e8, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.049381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.145525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 984/1976] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:31 2022] PUT /identity/v3/projects/8fd8eac40af54075afccaf924f4a87c8/users/a66a3ba723434c589dfd63e3d5fc11e8/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:32.146561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.148344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.156755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f1be7a73dc341c98eed3bac9ccd18ba', 'request_id': 'req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.164152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f1be7a73dc341c98eed3bac9ccd18ba', 'request_id': 'req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.215222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db2e18-cda5-4ee0-a677-a54e74cc2ac3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.263300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e0e2e-a781-48f0-bfc5-c2ef26623701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.263956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0e2e-a781-48f0-bfc5-c2ef26623701 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.264235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0e2e-a781-48f0-bfc5-c2ef26623701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.264566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0e2e-a781-48f0-bfc5-c2ef26623701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.268153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f1be7a73dc341c98eed3bac9ccd18ba', 'request_id': 'req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.281507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] The token cache is being invalidate because project 5f1be7a73dc341c98eed3bac9ccd18ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:32.281808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.282572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.284196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 993/1977] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:31 2022] DELETE /identity/v3/projects/5f1be7a73dc341c98eed3bac9ccd18ba => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:32.284999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.286276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.326235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0e2e-a781-48f0-bfc5-c2ef26623701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.327698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0e2e-a781-48f0-bfc5-c2ef26623701 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.337808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 985/1978] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:32.339073 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.339392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.339648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.350998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dd9b8-0fdb-4962-b5a1-2d5617c1b91d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.359693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:32.360947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.363415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6550a-0301-4068-b71f-9ad640c520de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.364678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6550a-0301-4068-b71f-9ad640c520de admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.365223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6550a-0301-4068-b71f-9ad640c520de admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.365655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6550a-0301-4068-b71f-9ad640c520de admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.366816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6550a-0301-4068-b71f-9ad640c520de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.369034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6550a-0301-4068-b71f-9ad640c520de admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.372630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 986/1979] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:32.372630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.380050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.386077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 994/1980] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:32.386077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab345b8-6790-4b7e-99ed-48e2dea76253 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.386077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab345b8-6790-4b7e-99ed-48e2dea76253 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.386077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab345b8-6790-4b7e-99ed-48e2dea76253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.393526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23edcb-f469-4f4f-a680-228a319dcb95 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.398170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.400523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.400523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.400523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] PATH_INFO: `/v3/projects/e1cb2fa9479c4651a20586958565af6b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.400909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1cb2fa9479c4651a20586958565af6b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.402834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.408735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0ab345b8-6790-4b7e-99ed-48e2dea76253 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:32.409976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0ab345b8-6790-4b7e-99ed-48e2dea76253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.419710 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 995/1981] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:32.423479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab345b8-6790-4b7e-99ed-48e2dea76253 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.430858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab345b8-6790-4b7e-99ed-48e2dea76253 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.436059 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.437531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.437958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.438678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] PATH_INFO: `/v3/projects/8fd8eac40af54075afccaf924f4a87c8/users/a66a3ba723434c589dfd63e3d5fc11e8/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.440920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd8eac40af54075afccaf924f4a87c8, user_id=a66a3ba723434c589dfd63e3d5fc11e8, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.443685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.454261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 996/1982] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:32 2022] PUT /identity/v3/projects/8fd8eac40af54075afccaf924f4a87c8/users/a66a3ba723434c589dfd63e3d5fc11e8/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:32.456585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.499828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:32.515320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1cb2fa9479c4651a20586958565af6b', 'request_id': 'req-d2e9d564-a0fc-4593-aa36-423eebd68589'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.517726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e2e8-7274-4948-b32c-9e62bac4f56d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.520659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1cb2fa9479c4651a20586958565af6b', 'request_id': 'req-d2e9d564-a0fc-4593-aa36-423eebd68589'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.527109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1cb2fa9479c4651a20586958565af6b', 'request_id': 'req-d2e9d564-a0fc-4593-aa36-423eebd68589'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.572840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132a85f-f27e-4b1a-bc2b-2675b503f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.573532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132a85f-f27e-4b1a-bc2b-2675b503f154 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.573814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132a85f-f27e-4b1a-bc2b-2675b503f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.574093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132a85f-f27e-4b1a-bc2b-2675b503f154 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.574810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132a85f-f27e-4b1a-bc2b-2675b503f154 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:32.575117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132a85f-f27e-4b1a-bc2b-2675b503f154 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.576727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132a85f-f27e-4b1a-bc2b-2675b503f154 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.588680 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 997/1983] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:32.589995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c05331-6ca5-46ff-b451-0826cf34bfa1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.590287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c05331-6ca5-46ff-b451-0826cf34bfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.590565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c05331-6ca5-46ff-b451-0826cf34bfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.594573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] The token cache is being invalidate because project e1cb2fa9479c4651a20586958565af6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:32.594793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:32.594903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.595640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:32.597415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 987/1984] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:32 2022] DELETE /identity/v3/projects/e1cb2fa9479c4651a20586958565af6b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:32.599954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.602907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.616659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9d564-a0fc-4593-aa36-423eebd68589 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.621348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b7ed6-d4de-4e60-a001-7d7f60b49ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.621550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c5c05331-6ca5-46ff-b451-0826cf34bfa1 None None] MFA Rules not processed for user `a66a3ba723434c589dfd63e3d5fc11e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:32.622009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b7ed6-d4de-4e60-a001-7d7f60b49ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.622294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b7ed6-d4de-4e60-a001-7d7f60b49ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.622574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b7ed6-d4de-4e60-a001-7d7f60b49ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.622739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c5c05331-6ca5-46ff-b451-0826cf34bfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.623268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b7ed6-d4de-4e60-a001-7d7f60b49ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.625694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b7ed6-d4de-4e60-a001-7d7f60b49ec5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.690996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 998/1985] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:32.692425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.692744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.693024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.711138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:32.712326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.720105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 999/1986] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:32.720884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.722077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.774631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 988/1987] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:32.774830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5e712-9b1e-49a8-b1d9-f4a008dddc3e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.774944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5e712-9b1e-49a8-b1d9-f4a008dddc3e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.775223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5e712-9b1e-49a8-b1d9-f4a008dddc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.777287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ed468-0a50-43b5-b0ed-448d048604ef None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.792653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-65c5e712-9b1e-49a8-b1d9-f4a008dddc3e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:32.793900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-65c5e712-9b1e-49a8-b1d9-f4a008dddc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:32.829849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.830500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.830785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.831059 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.856955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 989/1988] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:32.857849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5e712-9b1e-49a8-b1d9-f4a008dddc3e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.867942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5e712-9b1e-49a8-b1d9-f4a008dddc3e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.872701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.873362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.873671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.873981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.874666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.876840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.897405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.898840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.909026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1000/1989] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:32 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:32.909811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.919378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576ce70-19ef-4142-9958-83bbeb56091e service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.923949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.924631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.924923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.925203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.925925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:32.926237 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.927807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.939463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1001/1990] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:32.945116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.955357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3bee5-18e1-47ad-b925-3a984cdbb53c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:32.965975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:32.966837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:32.967427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:32.968452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:32.972712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 990/1991] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:32.972851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:32.972851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:32.988817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:32.999829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34460a0-8f6b-4952-a4d4-dddd266803a5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.004825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.005570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.005857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.006137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.006867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.008517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.019099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 991/1992] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:33.022997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.032492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe238a3-3c57-4250-b230-3a6656a45ec2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.036987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.037639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.037921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.038198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.038898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.040501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.051882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 992/1993] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:33.055986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.066218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79430115-e9e5-4e07-a44a-736dc956619d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.071191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113f15b-4c23-49d6-9675-643bf6ea7a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.071760 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113f15b-4c23-49d6-9675-643bf6ea7a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.072053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113f15b-4c23-49d6-9675-643bf6ea7a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.072332 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113f15b-4c23-49d6-9675-643bf6ea7a68 admin admin] PATH_INFO: `/v3/projects/85b76b83c1dd47928d7ae3e5b5df744e/users/099b7d203f2b4ea399d2c3097a0d9c1d/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.081369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1002/1994] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:32 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:33.086892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.095466 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113f15b-4c23-49d6-9675-643bf6ea7a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b76b83c1dd47928d7ae3e5b5df744e, user_id=099b7d203f2b4ea399d2c3097a0d9c1d, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.098238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113f15b-4c23-49d6-9675-643bf6ea7a68 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.103188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:33.157348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9cb14-03cf-4e7a-a7ec-037d0702f905 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.159139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.159990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.161068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.161068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.161510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.163592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.187783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 993/1995] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:33 2022] PUT /identity/v3/projects/85b76b83c1dd47928d7ae3e5b5df744e/users/099b7d203f2b4ea399d2c3097a0d9c1d/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:33.198898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17ed36-ce83-4728-b9be-0b7e5c0f0ed6 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.199365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17ed36-ce83-4728-b9be-0b7e5c0f0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.199969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17ed36-ce83-4728-b9be-0b7e5c0f0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.229264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cd17ed36-ce83-4728-b9be-0b7e5c0f0ed6 None None] MFA Rules not processed for user `099b7d203f2b4ea399d2c3097a0d9c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:33.230505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cd17ed36-ce83-4728-b9be-0b7e5c0f0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:33.298536 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1003/1996] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/users => generated 285 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:33.304002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.304851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 994/1997] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:33.309003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17ed36-ce83-4728-b9be-0b7e5c0f0ed6 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.324978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aef62-ea48-4487-9151-a1f3e1f372e2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.378925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17ed36-ce83-4728-b9be-0b7e5c0f0ed6 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.383679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.384342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.384659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.384940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.385643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.387554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.395124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.395852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.396147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.396422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.397168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.398797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.401818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:33.410829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1004/1998] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:33.411646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.413351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:33.469067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de15697-1d88-4bf5-be6a-509b81983166 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.472843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.473487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.473775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.476398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.476398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:33.480219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 995/1999] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:33.480953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.534204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.535668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.544245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439571db-63e4-40e3-afb9-c7d2aa2c8083 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.544740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1005/2000] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:33.547608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.549959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.550803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.551141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.551415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.552159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.554373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.558584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dbaca-c6a4-44f3-8120-9e8968c5acc0 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.563644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.564346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.567249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.567249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.567249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.567676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 996/2001] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:33.567754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.570883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.581165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ee748-390b-4023-a8f9-bbea07eda12f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.585883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eaa359-b4e1-49e8-8bad-61e056234d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.590140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaa359-b4e1-49e8-8bad-61e056234d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.590140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaa359-b4e1-49e8-8bad-61e056234d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.590140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eaa359-b4e1-49e8-8bad-61e056234d43 admin admin] PATH_INFO: `/v3/projects/37e75cddf66a406dadf9809be5e34d8f/users/c422945a6b2e4613bcc46247a742c337/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.602054 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eaa359-b4e1-49e8-8bad-61e056234d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e75cddf66a406dadf9809be5e34d8f, user_id=c422945a6b2e4613bcc46247a742c337, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.604778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eaa359-b4e1-49e8-8bad-61e056234d43 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.692194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1006/2002] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:33.697240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.707484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8e2df-b673-42ab-bd3a-86344ec48c8f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.712132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.712832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.713123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.716290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.716290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.716290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.720165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 997/2003] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:33 2022] PUT /identity/v3/projects/37e75cddf66a406dadf9809be5e34d8f/users/c422945a6b2e4613bcc46247a742c337/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:33.725914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1007/2004] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:33.730120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.730515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee10ff6-61fd-4214-bbe6-a729e1e5dd0f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.730807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee10ff6-61fd-4214-bbe6-a729e1e5dd0f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.731097 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee10ff6-61fd-4214-bbe6-a729e1e5dd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.740997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063c68a-8aec-48f0-b06f-df787bbd2930 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.760216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bee10ff6-61fd-4214-bbe6-a729e1e5dd0f None None] MFA Rules not processed for user `c422945a6b2e4613bcc46247a742c337`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:33.762411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bee10ff6-61fd-4214-bbe6-a729e1e5dd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:33.798840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.799585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.799691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:33.802400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.802400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.802400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.802770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.813327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1008/2005] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:33.817635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.827118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ddfb8-a9f1-41a2-a4e8-534a9a59fd67 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.831783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2929b9-2789-4aa4-a530-3d2949b9a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.832695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2929b9-2789-4aa4-a530-3d2949b9a754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.832990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2929b9-2789-4aa4-a530-3d2949b9a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.833274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2929b9-2789-4aa4-a530-3d2949b9a754 admin admin] PATH_INFO: `/v3/projects/da7e7a66a67e4003870a94c7e708dc6b/users/6773ed4d3bcd42f9b3f8a5701b8701c5/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.841467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 998/2006] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:33.845976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee10ff6-61fd-4214-bbe6-a729e1e5dd0f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:33.848675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2929b9-2789-4aa4-a530-3d2949b9a754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7e7a66a67e4003870a94c7e708dc6b, user_id=6773ed4d3bcd42f9b3f8a5701b8701c5, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.851370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2929b9-2789-4aa4-a530-3d2949b9a754 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.856534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee10ff6-61fd-4214-bbe6-a729e1e5dd0f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:33.861592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:33.862248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.866655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.866655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.866655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:33.866655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:33.963655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1009/2007] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:33 2022] PUT /identity/v3/projects/da7e7a66a67e4003870a94c7e708dc6b/users/6773ed4d3bcd42f9b3f8a5701b8701c5/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:33.975827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba4fef-73d4-419e-9ba3-0b3373a9c751 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:33.976119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba4fef-73d4-419e-9ba3-0b3373a9c751 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:33.976400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba4fef-73d4-419e-9ba3-0b3373a9c751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:33.993128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 999/2008] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:33.994094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.004206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d6ba4fef-73d4-419e-9ba3-0b3373a9c751 None None] MFA Rules not processed for user `6773ed4d3bcd42f9b3f8a5701b8701c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:34.005440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d6ba4fef-73d4-419e-9ba3-0b3373a9c751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:34.057345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ae0de-6f88-40ac-bf63-436e487307ee admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.062973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.063724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.064020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.064317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.065089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.067335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.071491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1010/2009] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:34.076077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba4fef-73d4-419e-9ba3-0b3373a9c751 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.086692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba4fef-73d4-419e-9ba3-0b3373a9c751 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.091437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.092085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.092901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.093180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.093889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.095777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.167559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1011/2011] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:34 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:34.167714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1000/2011] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:33 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:34.174941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.175084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.187177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1e16f-ed2c-413e-99a7-314b4c67a584 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.229077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:34.253803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.254651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.254937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.255290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.255996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.259381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.276362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd2ce8-29c4-43a5-941e-9589d73cbd20 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.282257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.283096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.283384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.285639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.285639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.286234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.304614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1012/2012] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:34.310528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.322885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9af77-4949-494f-83f7-dd7c1065c391 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.328453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.329082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.329367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.329645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] PATH_INFO: `/v3/projects/028db8ad74174f09afdfa0c18342e38d/users/4d811184cb0d44d3866d76f884f56ca8/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.345690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] RBAC: Authorizing `identity:create_grant(project_id=028db8ad74174f09afdfa0c18342e38d, user_id=4d811184cb0d44d3866d76f884f56ca8, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.348756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.402487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1001/2013] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:34 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:34.404107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.413689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bf4f5-ee09-418b-9c41-85cd0151ffa5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.418255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.418899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.419185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.419589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.421122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.422943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.425547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:34.431142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1013/2014] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:34 2022] PUT /identity/v3/projects/028db8ad74174f09afdfa0c18342e38d/users/4d811184cb0d44d3866d76f884f56ca8/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:34.434105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.435487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1002/2015] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:34.439716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.449385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f08b9-47e2-4014-b3fc-fec51313e56a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.496363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8f19a-e626-4268-8cf2-9347a813802f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.501833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.502510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.502801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.503086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.503822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.505490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.506457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.507356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.507833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.508064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] PATH_INFO: `/v3/projects/09343de02e174191894a09ce364f9374/users/f4501a5c12654e388b4848a443276b7d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.512718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:34.518894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1014/2016] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:34.522594 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.523673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09343de02e174191894a09ce364f9374, user_id=f4501a5c12654e388b4848a443276b7d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.526462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.532101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdae5a-e197-4b29-9778-b2384c86b3ff admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.536868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fffb72-914a-4562-8c33-68c4e989a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.537733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fffb72-914a-4562-8c33-68c4e989a5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.538015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fffb72-914a-4562-8c33-68c4e989a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.538293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fffb72-914a-4562-8c33-68c4e989a5a2 admin admin] PATH_INFO: `/v3/projects/028db8ad74174f09afdfa0c18342e38d/users/4d811184cb0d44d3866d76f884f56ca8/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.540323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fffb72-914a-4562-8c33-68c4e989a5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=028db8ad74174f09afdfa0c18342e38d, user_id=4d811184cb0d44d3866d76f884f56ca8, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.542992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fffb72-914a-4562-8c33-68c4e989a5a2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.553815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1015/2017] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:34 2022] PUT /identity/v3/projects/028db8ad74174f09afdfa0c18342e38d/users/4d811184cb0d44d3866d76f884f56ca8/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:34.567077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3bcc8-ba9a-45ef-9de8-80c4e500ca4d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.567315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3bcc8-ba9a-45ef-9de8-80c4e500ca4d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.567617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3bcc8-ba9a-45ef-9de8-80c4e500ca4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.595796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-23f3bcc8-ba9a-45ef-9de8-80c4e500ca4d None None] MFA Rules not processed for user `4d811184cb0d44d3866d76f884f56ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:34.597057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-23f3bcc8-ba9a-45ef-9de8-80c4e500ca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:34.646092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1003/2018] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:34 2022] PUT /identity/v3/projects/09343de02e174191894a09ce364f9374/users/f4501a5c12654e388b4848a443276b7d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:34.650081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.669532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1016/2019] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:34.680056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3bcc8-ba9a-45ef-9de8-80c4e500ca4d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.716357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49bc70-1ea3-4b49-9dbd-96112c5bfc11 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.721315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.721987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.722274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.722553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.723318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.724969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.734788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1004/2020] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:34.737665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3bcc8-ba9a-45ef-9de8-80c4e500ca4d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.739340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.749908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bf3a-a8d1-4d7d-adb9-6a440316c9f4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.754888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19b11a-378a-4cfd-a97a-4e7ad4a7c517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.755808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b11a-378a-4cfd-a97a-4e7ad4a7c517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.756096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b11a-378a-4cfd-a97a-4e7ad4a7c517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.756370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b11a-378a-4cfd-a97a-4e7ad4a7c517 admin admin] PATH_INFO: `/v3/projects/09343de02e174191894a09ce364f9374/users/f4501a5c12654e388b4848a443276b7d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.758412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19b11a-378a-4cfd-a97a-4e7ad4a7c517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09343de02e174191894a09ce364f9374, user_id=f4501a5c12654e388b4848a443276b7d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.761251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19b11a-378a-4cfd-a97a-4e7ad4a7c517 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.772330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1005/2021] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:34 2022] PUT /identity/v3/projects/09343de02e174191894a09ce364f9374/users/f4501a5c12654e388b4848a443276b7d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:34.782643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec0acb-d3d5-4d8f-af0f-cc640f35ee3b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.783376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec0acb-d3d5-4d8f-af0f-cc640f35ee3b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.783684 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec0acb-d3d5-4d8f-af0f-cc640f35ee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.786609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:34.788138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.788847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.789133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.789407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.811984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-12ec0acb-d3d5-4d8f-af0f-cc640f35ee3b None None] MFA Rules not processed for user `f4501a5c12654e388b4848a443276b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:34.813249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-12ec0acb-d3d5-4d8f-af0f-cc640f35ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:34.838137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:34.852531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.854022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.863892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1017/2022] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:34 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:34.882378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1006/2023] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:34.899505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:34.908068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c7fd1-f3b8-43d9-9e8e-7f47cc07ce4d service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:34.958299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:34.958942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:34.959225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:34.959501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:34.968677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:34.970093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:34.979461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1018/2024] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:35.063700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:35.065836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec0acb-d3d5-4d8f-af0f-cc640f35ee3b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:35.075475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec0acb-d3d5-4d8f-af0f-cc640f35ee3b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:35.079788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437e8ee-d893-41c8-90f5-3ee3b6227569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:35.080453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437e8ee-d893-41c8-90f5-3ee3b6227569 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:35.080755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437e8ee-d893-41c8-90f5-3ee3b6227569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:35.081056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437e8ee-d893-41c8-90f5-3ee3b6227569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:35.126407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:35.141073 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437e8ee-d893-41c8-90f5-3ee3b6227569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:35.142593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437e8ee-d893-41c8-90f5-3ee3b6227569 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:35.151479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1007/2025] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:35.251611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:36.380614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:36.383018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:36.464289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e81be-e574-4f1f-b98d-98a8f12eae96 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:36.472494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:36.472494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:36.472494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:36.472494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] PATH_INFO: `/v3/users/642f415de5c04294a8094a3420ba8523` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:36.472983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=642f415de5c04294a8094a3420ba8523)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:36.474624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:36.814111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642f415de5c04294a8094a3420ba8523'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:36.814789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642f415de5c04294a8094a3420ba8523'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:36.815394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642f415de5c04294a8094a3420ba8523'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:36.827328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642f415de5c04294a8094a3420ba8523'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:36.925529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642f415de5c04294a8094a3420ba8523'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:36.938790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:36.943771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1019/2026] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:36 2022] DELETE /identity/v3/users/642f415de5c04294a8094a3420ba8523 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:36.948399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:36.948871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:36.948978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:36.968929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:36.970169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:37.032855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1008/2027] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:37.045235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:37.049328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:37.136962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f6969-79ab-4e73-8fd2-6988a13e4a8e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:37.141092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbcad6-6fb4-4732-aa62-6b90f4fa753c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:37.141753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbcad6-6fb4-4732-aa62-6b90f4fa753c service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:37.142040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbcad6-6fb4-4732-aa62-6b90f4fa753c service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:37.142318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbcad6-6fb4-4732-aa62-6b90f4fa753c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:37.159107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbcad6-6fb4-4732-aa62-6b90f4fa753c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:37.160772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbcad6-6fb4-4732-aa62-6b90f4fa753c service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:37.172065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1020/2028] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:37.703899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:37.705374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:37.717337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f7196-9781-4f5b-ac2b-a3ca40c5a01f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:37.722655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:37.723535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:37.724050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:37.724154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] PATH_INFO: `/v3/projects/fc7ae0930e6d4affb6c456846594e513` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:37.725317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7ae0930e6d4affb6c456846594e513)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:37.727231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:37.846174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7ae0930e6d4affb6c456846594e513', 'request_id': 'req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:37.851937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7ae0930e6d4affb6c456846594e513', 'request_id': 'req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:37.858100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7ae0930e6d4affb6c456846594e513', 'request_id': 'req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:37.947823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] The token cache is being invalidate because project fc7ae0930e6d4affb6c456846594e513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:37.948050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:37.948712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-988cde71-f5b0-4bfd-9fd4-f5feae2a756d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:37.950325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1009/2029] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:37 2022] DELETE /identity/v3/projects/fc7ae0930e6d4affb6c456846594e513 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:37.955870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79eef3-bb92-4bde-8b8c-9e9d5296867a None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:37.956278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79eef3-bb92-4bde-8b8c-9e9d5296867a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:37.956494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79eef3-bb92-4bde-8b8c-9e9d5296867a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:37.975051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-df79eef3-bb92-4bde-8b8c-9e9d5296867a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:37.985172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-df79eef3-bb92-4bde-8b8c-9e9d5296867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:38.056603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1021/2030] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:38.065552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4aa2e-1a8d-48cd-90bc-46609daa7826 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.065849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4aa2e-1a8d-48cd-90bc-46609daa7826 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.066134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4aa2e-1a8d-48cd-90bc-46609daa7826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.085672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-30d4aa2e-1a8d-48cd-90bc-46609daa7826 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:38.086982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-30d4aa2e-1a8d-48cd-90bc-46609daa7826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:38.095232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1010/2031] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:38.099709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79eef3-bb92-4bde-8b8c-9e9d5296867a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:38.110628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79eef3-bb92-4bde-8b8c-9e9d5296867a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:38.115333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:38.116021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.116316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.116630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.117355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:38.124099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:38.124099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:38.131163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1022/2032] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:50:38.135589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4aa2e-1a8d-48cd-90bc-46609daa7826 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:38.145283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4aa2e-1a8d-48cd-90bc-46609daa7826 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:38.149880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:38.150523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.150805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.151080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.151847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:38.153765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:38.289659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1011/2033] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:38 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:38.290877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:38.301821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da046f4-b12c-4f40-9112-4bc77e3e7c69 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:38.368772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:38.369988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.370432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.370860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.371956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:38.379218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:38.461815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1023/2034] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:38 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:38.470331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:38.481734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aec23-3543-4575-89d8-9d0f0a475034 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:38.487441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:38.488256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.488833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.489093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.489868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:38.491465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:38.504477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1012/2035] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:38.509029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:38.520918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00723d-5361-4c23-9b5e-7a5c6e35e1d1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:38.525848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4b85e-771b-4317-93d5-3f10b29ec7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:38.526516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4b85e-771b-4317-93d5-3f10b29ec7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.526802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4b85e-771b-4317-93d5-3f10b29ec7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.527080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4b85e-771b-4317-93d5-3f10b29ec7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.527840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4b85e-771b-4317-93d5-3f10b29ec7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:38.529461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4b85e-771b-4317-93d5-3f10b29ec7ea admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:38.541145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1024/2036] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:38.545311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:38.556105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2dfb6-d0ab-4ae0-bf3e-76ee22862ab0 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:38.561445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:38.562335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.562621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.563003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] PATH_INFO: `/v3/projects/2b7907f113c54585af1d8411b4757897/users/6d9891dce2124828b706bef92e7a12ac/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.581013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7907f113c54585af1d8411b4757897, user_id=6d9891dce2124828b706bef92e7a12ac, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:38.583670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:38.657247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1013/2037] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:38 2022] PUT /identity/v3/projects/2b7907f113c54585af1d8411b4757897/users/6d9891dce2124828b706bef92e7a12ac/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:38.668339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.668659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.668941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.681831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:38.700332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] MFA Rules not processed for user `6d9891dce2124828b706bef92e7a12ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:38.701659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:38.767153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1025/2038] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:38.778612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:38.780898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:38.786049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:38.837446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e3223-765b-4be3-8f6f-454451547b43 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:38.841104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19214c-0e69-467f-bc80-dd0f5e9f7c7b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:38.841746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19214c-0e69-467f-bc80-dd0f5e9f7c7b service glance] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:38.842029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19214c-0e69-467f-bc80-dd0f5e9f7c7b service glance] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:38.842305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19214c-0e69-467f-bc80-dd0f5e9f7c7b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:38.854882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19214c-0e69-467f-bc80-dd0f5e9f7c7b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:38.856483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19214c-0e69-467f-bc80-dd0f5e9f7c7b service glance] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:38.869708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1014/2039] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:39.149326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:39.150780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:39.214774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029597c-e478-455e-a49d-d42b5d22e212 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:39.219291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:39.220160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:39.220478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:39.220763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] PATH_INFO: `/v3/users/c422945a6b2e4613bcc46247a742c337` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:39.221940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c422945a6b2e4613bcc46247a742c337)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:39.223851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:39.507040 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c422945a6b2e4613bcc46247a742c337'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:39.508116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c422945a6b2e4613bcc46247a742c337'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:39.508621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c422945a6b2e4613bcc46247a742c337'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:39.519566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c422945a6b2e4613bcc46247a742c337'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:39.628297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c422945a6b2e4613bcc46247a742c337'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:39.658972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1026/2040] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:39 2022] DELETE /identity/v3/users/c422945a6b2e4613bcc46247a742c337 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:39.663399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:39.663883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:39.664001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:39.682933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:39.684668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:39.748727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1015/2041] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:39.759145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:39.760922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:39.786455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:39.829834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a08eb-c367-4434-85e9-4aabafe61820 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:39.833289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:39.833924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:39.834205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:39.834477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:39.843389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:39.844928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:39.854351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1027/2042] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:40.245464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:40.247216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:40.261504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd08a3e-be6f-4e5a-ba1e-106af499caf2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:40.267980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:40.268891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:40.269181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:40.269505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] PATH_INFO: `/v3/projects/37e75cddf66a406dadf9809be5e34d8f` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:40.270840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e75cddf66a406dadf9809be5e34d8f)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:40.282632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:40.446101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e75cddf66a406dadf9809be5e34d8f', 'request_id': 'req-c8e14981-8f08-4d13-9616-5fb28b579a83'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.452377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e75cddf66a406dadf9809be5e34d8f', 'request_id': 'req-c8e14981-8f08-4d13-9616-5fb28b579a83'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.458719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e75cddf66a406dadf9809be5e34d8f', 'request_id': 'req-c8e14981-8f08-4d13-9616-5fb28b579a83'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.547749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] The token cache is being invalidate because project 37e75cddf66a406dadf9809be5e34d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:40.548039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.548713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.550401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1016/2043] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:40 2022] DELETE /identity/v3/projects/37e75cddf66a406dadf9809be5e34d8f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:40.554598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:40.565859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbee99-2800-40b1-a69b-6533f845197a service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:40.598108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:40.600840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:40.636217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:40.644391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:40.644391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:40.644391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:40.644391 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] PATH_INFO: `/v3/users/4d811184cb0d44d3866d76f884f56ca8` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:40.645246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d811184cb0d44d3866d76f884f56ca8)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:40.647187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:40.703016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e14981-8f08-4d13-9616-5fb28b579a83 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:40.711125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:40.712104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:40.712394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:40.712722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] PATH_INFO: `/v3/users/384026a0822a4d7fa04cde1d20c6822b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:40.714178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=384026a0822a4d7fa04cde1d20c6822b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:40.716330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:40.894406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d811184cb0d44d3866d76f884f56ca8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.895365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d811184cb0d44d3866d76f884f56ca8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.896067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d811184cb0d44d3866d76f884f56ca8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.903180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d811184cb0d44d3866d76f884f56ca8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.949682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384026a0822a4d7fa04cde1d20c6822b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.961158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384026a0822a4d7fa04cde1d20c6822b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.961829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384026a0822a4d7fa04cde1d20c6822b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.962446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384026a0822a4d7fa04cde1d20c6822b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:40.969689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384026a0822a4d7fa04cde1d20c6822b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:41.039819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-97b627d7-13e8-4464-8381-e27c8bb69397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d811184cb0d44d3866d76f884f56ca8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:41.059811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1028/2044] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:40 2022] DELETE /identity/v3/users/4d811184cb0d44d3866d76f884f56ca8 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:41.141132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1017/2045] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:40 2022] DELETE /identity/v3/users/384026a0822a4d7fa04cde1d20c6822b => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:41.145959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:41.146452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:41.146719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:41.166989 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:41.168328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:41.210249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:41.238845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1029/2046] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:41.249147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:41.250820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:41.324138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5127b-f300-4305-9b4a-35e8cd145746 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:41.327914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93046c5-1fc7-483a-862b-92fd14097492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:41.328602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93046c5-1fc7-483a-862b-92fd14097492 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:41.328890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93046c5-1fc7-483a-862b-92fd14097492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:41.329168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93046c5-1fc7-483a-862b-92fd14097492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:41.338768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93046c5-1fc7-483a-862b-92fd14097492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:41.340176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93046c5-1fc7-483a-862b-92fd14097492 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:41.350372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1018/2047] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:41.544508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:41.546375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:41.559492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c984-bc5c-4c47-964d-990ae0034751 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:41.565287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:41.566390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:41.567024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:41.567507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] PATH_INFO: `/v3/projects/028db8ad74174f09afdfa0c18342e38d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:41.569016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] RBAC: Authorizing `identity:delete_project(project_id=028db8ad74174f09afdfa0c18342e38d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:41.571389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:41.694798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '028db8ad74174f09afdfa0c18342e38d', 'request_id': 'req-0b5fff96-eef0-4df5-9ce3-d301c107d09c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:41.704784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '028db8ad74174f09afdfa0c18342e38d', 'request_id': 'req-0b5fff96-eef0-4df5-9ce3-d301c107d09c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:41.812530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '028db8ad74174f09afdfa0c18342e38d', 'request_id': 'req-0b5fff96-eef0-4df5-9ce3-d301c107d09c'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:41.840765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] The token cache is being invalidate because project 028db8ad74174f09afdfa0c18342e38d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:41.841207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:41.842037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0b5fff96-eef0-4df5-9ce3-d301c107d09c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:41.844153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1030/2048] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:41 2022] DELETE /identity/v3/projects/028db8ad74174f09afdfa0c18342e38d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:41.849128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dcec9-faea-4598-ae91-f4a778fc37a7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:41.849582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dcec9-faea-4598-ae91-f4a778fc37a7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:41.849686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dcec9-faea-4598-ae91-f4a778fc37a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:41.878095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d78dcec9-faea-4598-ae91-f4a778fc37a7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:41.879586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d78dcec9-faea-4598-ae91-f4a778fc37a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:41.934005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:41.954039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1019/2049] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:41.964172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:41.964172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:41.964357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:41.984311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:41.985588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:41.997555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1031/2050] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:41.998613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:41.998776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dcec9-faea-4598-ae91-f4a778fc37a7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.000021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:42.008812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dcec9-faea-4598-ae91-f4a778fc37a7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.016584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113d656-65ad-4d7b-9b84-e1f380bb26fb None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.020026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.020978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.021537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.021817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] PATH_INFO: `/v3/projects/64254ff60fcc48a2ba82aa7dafc35709` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.022078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.022963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.023074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64254ff60fcc48a2ba82aa7dafc35709)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.023426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.023803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.024689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:42.025011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.025175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.027213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.042153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1020/2051] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:42.046397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.058845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162a31c-2fd4-450c-97cb-8d82466ae584 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.073253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.074134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.074416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.074708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.075422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.077911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.207218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64254ff60fcc48a2ba82aa7dafc35709', 'request_id': 'req-6bd7bb41-bccf-4a82-bccc-2e4203654f31'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:42.214228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64254ff60fcc48a2ba82aa7dafc35709', 'request_id': 'req-6bd7bb41-bccf-4a82-bccc-2e4203654f31'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:42.219921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1021/2052] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:42.220958 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.284474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c1259-771b-4b6d-bf3c-cd2742764e25 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.289542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.290229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.290517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.290845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.291516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.293568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.308485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64254ff60fcc48a2ba82aa7dafc35709', 'request_id': 'req-6bd7bb41-bccf-4a82-bccc-2e4203654f31'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:42.322811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] The token cache is being invalidate because project 64254ff60fcc48a2ba82aa7dafc35709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:42.323251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:42.324082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6bd7bb41-bccf-4a82-bccc-2e4203654f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:42.325862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1032/2053] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:41 2022] DELETE /identity/v3/projects/64254ff60fcc48a2ba82aa7dafc35709 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:42.332565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2452f3-f2a3-4bcf-838b-ffc70e7ca3db None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.332933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2452f3-f2a3-4bcf-838b-ffc70e7ca3db None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.333293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2452f3-f2a3-4bcf-838b-ffc70e7ca3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.351481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-8b2452f3-f2a3-4bcf-838b-ffc70e7ca3db None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:42.352806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8b2452f3-f2a3-4bcf-838b-ffc70e7ca3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:42.361147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1033/2054] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:42.368887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135b54-6be2-414c-b35f-85d043a93109 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.369181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135b54-6be2-414c-b35f-85d043a93109 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.369466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00135b54-6be2-414c-b35f-85d043a93109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.389147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-00135b54-6be2-414c-b35f-85d043a93109 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:42.390994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-00135b54-6be2-414c-b35f-85d043a93109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:42.405607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1034/2055] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:42.409759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00135b54-6be2-414c-b35f-85d043a93109 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.416112 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1022/2056] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:42.418041 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.419227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00135b54-6be2-414c-b35f-85d043a93109 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.420636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:42.424193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.425017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.425422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.425711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.426446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:42.426761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.428359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.437432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af7a31-014a-42ea-a676-3ae5a8801942 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.441400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1035/2057] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:42.445702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.446575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.447308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.447664 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.448044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.448884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.450626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.456396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20966c6c-aed8-4023-8baf-a812605b2100 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.462862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.463578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.463920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.466290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1023/2058] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:42.466407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.467201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.469241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.470561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.482249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:42.487272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e96b8-cd00-4a69-af9d-7fe6e8f939a8 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.493968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.494622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.494907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.495189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.495936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.497589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.512223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1024/2059] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:42.513316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.523307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51533b-2c13-4462-9dcb-e912c69477e2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.529316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1fa58-5679-4565-8eae-f16d31a10ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.530216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1fa58-5679-4565-8eae-f16d31a10ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.530498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1fa58-5679-4565-8eae-f16d31a10ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.530776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1fa58-5679-4565-8eae-f16d31a10ea3 admin admin] PATH_INFO: `/v3/projects/170e06a5f54b4570a24b230c7eefcb59/users/fab4b74ababf44f88748a98ff67c73a2/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.547246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1fa58-5679-4565-8eae-f16d31a10ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170e06a5f54b4570a24b230c7eefcb59, user_id=fab4b74ababf44f88748a98ff67c73a2, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.550114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1fa58-5679-4565-8eae-f16d31a10ea3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.659624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1036/2060] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:42.663709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.693191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1025/2061] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:42 2022] PUT /identity/v3/projects/170e06a5f54b4570a24b230c7eefcb59/users/fab4b74ababf44f88748a98ff67c73a2/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:42.703706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6173c86-2162-4fa2-88ef-bfee495028c0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.703996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6173c86-2162-4fa2-88ef-bfee495028c0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.704276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6173c86-2162-4fa2-88ef-bfee495028c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.732618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1747217-3c7d-4f8f-8a71-5584efa477a7 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.734361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a6173c86-2162-4fa2-88ef-bfee495028c0 None None] MFA Rules not processed for user `fab4b74ababf44f88748a98ff67c73a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:42.735574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a6173c86-2162-4fa2-88ef-bfee495028c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:42.818076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.818780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.819167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.819523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.820312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.826090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.833860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1026/2062] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:42.837042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6173c86-2162-4fa2-88ef-bfee495028c0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:42.850025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6173c86-2162-4fa2-88ef-bfee495028c0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:42.855021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:42.855801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:42.856104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:42.856390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:42.860249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:42.860249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:42.992124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1037/2063] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/users => generated 287 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:42.997415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.016062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837979b9-db3c-421c-882c-5e81ea281286 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.020116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1027/2064] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:42 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:43.036256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.125213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.126005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.126818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.126818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.127374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.129763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.147162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7861e3-c947-4624-ab0d-8ce94222d018 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.155920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1038/2065] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:43.156995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.159072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:43.166767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.166767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.166767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.166767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.167193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.167193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.167352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ec1f1-2c37-4ec0-9c18-c375c46e38c1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.174549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.175578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.175956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.176298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.177520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.179435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.193812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1039/2066] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:43.198332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.209480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d64687-5839-455c-b506-3369c3901224 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.214597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.215525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.215847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.216130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] PATH_INFO: `/v3/projects/0bcdae79468045d0b5d773d8caf238e6/users/694570bd24c74fd19e80bd822048067b/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.230866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcdae79468045d0b5d773d8caf238e6, user_id=694570bd24c74fd19e80bd822048067b, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.233646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.280037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1028/2067] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:43 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:43.280930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.283290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:43.296015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0357ed-e72e-4a30-89c6-891c7bc63fd3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.300710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.301567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.301862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.302116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] PATH_INFO: `/v3/users/099b7d203f2b4ea399d2c3097a0d9c1d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.303303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] RBAC: Authorizing `identity:delete_user(user_id=099b7d203f2b4ea399d2c3097a0d9c1d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.305361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.388455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1040/2068] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:43 2022] PUT /identity/v3/projects/0bcdae79468045d0b5d773d8caf238e6/users/694570bd24c74fd19e80bd822048067b/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:43.389450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.449523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e9400-eb75-4c76-9e38-6491849d2138 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.454475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecfb51b-48cd-455d-8298-11600dc0359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.455169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfb51b-48cd-455d-8298-11600dc0359f admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.455620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfb51b-48cd-455d-8298-11600dc0359f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.455743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfb51b-48cd-455d-8298-11600dc0359f admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.456581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfb51b-48cd-455d-8298-11600dc0359f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.458200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfb51b-48cd-455d-8298-11600dc0359f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.469756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1041/2069] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:43.471006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c27d-a299-4ed8-a5e9-9e0dd0574259 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.471295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c27d-a299-4ed8-a5e9-9e0dd0574259 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.471581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2c27d-a299-4ed8-a5e9-9e0dd0574259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.505727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-90b2c27d-a299-4ed8-a5e9-9e0dd0574259 None None] MFA Rules not processed for user `694570bd24c74fd19e80bd822048067b`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:43.508230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-90b2c27d-a299-4ed8-a5e9-9e0dd0574259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:43.571626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099b7d203f2b4ea399d2c3097a0d9c1d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:43.574211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1042/2070] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:43.575129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2c27d-a299-4ed8-a5e9-9e0dd0574259 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.583923 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099b7d203f2b4ea399d2c3097a0d9c1d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:43.584829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099b7d203f2b4ea399d2c3097a0d9c1d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:43.585516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2c27d-a299-4ed8-a5e9-9e0dd0574259 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.585650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099b7d203f2b4ea399d2c3097a0d9c1d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:43.590463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd804c73-faba-49fa-87ad-efdf2b09dd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.591234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd804c73-faba-49fa-87ad-efdf2b09dd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.591525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd804c73-faba-49fa-87ad-efdf2b09dd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.591807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd804c73-faba-49fa-87ad-efdf2b09dd88 admin admin] PATH_INFO: `/v3/projects/8242a850b34a4532a8de58d321843a40/users/66421b04cccf435f909d0528ce4016ae/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.594769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099b7d203f2b4ea399d2c3097a0d9c1d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:43.608937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd804c73-faba-49fa-87ad-efdf2b09dd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8242a850b34a4532a8de58d321843a40, user_id=66421b04cccf435f909d0528ce4016ae, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.611797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd804c73-faba-49fa-87ad-efdf2b09dd88 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.691754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:43.731594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1029/2071] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:43 2022] DELETE /identity/v3/users/099b7d203f2b4ea399d2c3097a0d9c1d => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:43.731827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1043/2072] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:43 2022] PUT /identity/v3/projects/8242a850b34a4532a8de58d321843a40/users/66421b04cccf435f909d0528ce4016ae/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:43.732873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.734826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:43.736828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.737120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.737399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.754412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:43.755624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:43.800400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c62ee-dd22-482e-b65d-7bd8c28f704f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.804659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.805466 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.805886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.806232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.808211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:43.821868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:43.837834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1044/2073] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:43.838689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.839982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:43.858749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b93e1a-9baf-4f07-b6e3-6c1ae09c212d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.864134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.864999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.865293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.865576 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.866318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.868046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.878741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1045/2074] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:43.879549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.881440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:43.889144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.890834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.900888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1030/2075] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:43.901708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:43.913475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60207051-4c14-4807-8ef5-25aabfc1a045 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.923522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9b4fc-f983-4949-a0c6-02e06b68e99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:43.924569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9b4fc-f983-4949-a0c6-02e06b68e99f admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.924898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9b4fc-f983-4949-a0c6-02e06b68e99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.925183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9b4fc-f983-4949-a0c6-02e06b68e99f admin admin] PATH_INFO: `/v3/projects/8242a850b34a4532a8de58d321843a40/users/66421b04cccf435f909d0528ce4016ae/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.927279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9b4fc-f983-4949-a0c6-02e06b68e99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8242a850b34a4532a8de58d321843a40, user_id=66421b04cccf435f909d0528ce4016ae, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:43.930025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9b4fc-f983-4949-a0c6-02e06b68e99f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:43.941514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1031/2076] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:43 2022] PUT /identity/v3/projects/8242a850b34a4532a8de58d321843a40/users/66421b04cccf435f909d0528ce4016ae/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:43.953006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:43.953369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:43.953717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:43.960821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c680d-66df-41e6-b099-b4e19bee8d3e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:43.989453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] MFA Rules not processed for user `66421b04cccf435f909d0528ce4016ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:43.991003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:44.032495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:44.033061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:44.033347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:44.033627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:44.070435 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1032/2077] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:44.089028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:44.090482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:44.103696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:44.105327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:44.116755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1046/2078] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:44.153795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a4311-d9ac-4595-a4d4-4e991d54f6ec None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:44.158319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:44.158990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:44.159278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:44.159565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:44.169172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:44.170658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:44.181414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1033/2079] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:44.519991 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:44.585535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:44.587066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:44.602178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac1bd-a184-4ecb-82d7-ac057b92f938 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:44.611731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:44.611731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:44.611731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:44.611731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] PATH_INFO: `/v3/projects/85b76b83c1dd47928d7ae3e5b5df744e` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:44.612571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b76b83c1dd47928d7ae3e5b5df744e)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:44.612812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:44.754280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b76b83c1dd47928d7ae3e5b5df744e', 'request_id': 'req-9fe1c84e-4296-4260-bcd1-976a1f63af2a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:44.762678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b76b83c1dd47928d7ae3e5b5df744e', 'request_id': 'req-9fe1c84e-4296-4260-bcd1-976a1f63af2a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:44.830338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b76b83c1dd47928d7ae3e5b5df744e', 'request_id': 'req-9fe1c84e-4296-4260-bcd1-976a1f63af2a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:44.848952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] The token cache is being invalidate because project 85b76b83c1dd47928d7ae3e5b5df744e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:44.849240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:44.849966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:44.851525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1047/2080] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:44 2022] DELETE /identity/v3/projects/85b76b83c1dd47928d7ae3e5b5df744e => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:44.859701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:44.866629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dfeda-4192-4798-ab5f-73c1c1a3e04a service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:44.923190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:44.924072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:44.924357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:44.924667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] PATH_INFO: `/v3/users/6773ed4d3bcd42f9b3f8a5701b8701c5` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:44.925866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6773ed4d3bcd42f9b3f8a5701b8701c5)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:44.927792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:45.090619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:45.092144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:45.105739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1c84e-4296-4260-bcd1-976a1f63af2a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:45.111463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:45.112417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:45.112724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:45.113000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] PATH_INFO: `/v3/users/8bbbcc05646b4f4e9a88e513b9ff0f69` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:45.114192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bbbcc05646b4f4e9a88e513b9ff0f69)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:45.116093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:45.197474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6773ed4d3bcd42f9b3f8a5701b8701c5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.209442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6773ed4d3bcd42f9b3f8a5701b8701c5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.210151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6773ed4d3bcd42f9b3f8a5701b8701c5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.210752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6773ed4d3bcd42f9b3f8a5701b8701c5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.217852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-770d70e1-c1f0-47ff-b858-1efb89427a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6773ed4d3bcd42f9b3f8a5701b8701c5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.245229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:45.316518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bbbcc05646b4f4e9a88e513b9ff0f69'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.317250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bbbcc05646b4f4e9a88e513b9ff0f69'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.317844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bbbcc05646b4f4e9a88e513b9ff0f69'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.324673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bbbcc05646b4f4e9a88e513b9ff0f69'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.338784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1034/2081] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:44 2022] DELETE /identity/v3/users/6773ed4d3bcd42f9b3f8a5701b8701c5 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:45.445147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bbbcc05646b4f4e9a88e513b9ff0f69'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:45.471491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1048/2082] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:45 2022] DELETE /identity/v3/users/8bbbcc05646b4f4e9a88e513b9ff0f69 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:45.471609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:45.471609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:45.471609 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:45.489554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:45.490780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:45.555478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1035/2083] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:45.570082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:45.577540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:45.700860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a0cec-c1e3-450f-a852-ab375e726c28 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:45.703318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:45.704802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:45.704936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:45.705466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:45.705754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:45.706031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:45.715737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:45.720414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:45.732355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1049/2084] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:45.777584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbed9b-24ee-47f4-aca7-b34d522809dd None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:45.783714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5659153-126a-4f42-a723-33a6e3f4403c tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66421b04cccf435f909d0528ce4016ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8242a850b34a4532a8de58d321843a40', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:45.784649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659153-126a-4f42-a723-33a6e3f4403c tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:45.785005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659153-126a-4f42-a723-33a6e3f4403c tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:45.785350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659153-126a-4f42-a723-33a6e3f4403c tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:45.786109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5659153-126a-4f42-a723-33a6e3f4403c tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:45.786430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5659153-126a-4f42-a723-33a6e3f4403c tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:45.788392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5659153-126a-4f42-a723-33a6e3f4403c tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:45.794255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:45.806912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1036/2085] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:45.807975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:45.819281 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547720c-45c5-498a-8187-c9662ea51d0a service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:45.824050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66421b04cccf435f909d0528ce4016ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8242a850b34a4532a8de58d321843a40', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:45.824807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:45.825142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:45.825426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:45.826138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:45.828066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:45.854111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:45.854339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:45.854579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:45.876173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] MFA Rules not processed for user `694570bd24c74fd19e80bd822048067b`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:45.877591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:45.951428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1037/2086] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:45.951428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:45.951428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:45.975215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f51456-c29f-43ad-8986-5b3e7d6faefa None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:45.980115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:45.980819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:45.981153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:45.981436 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] PATH_INFO: `/v3/projects/da7e7a66a67e4003870a94c7e708dc6b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:45.982794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7e7a66a67e4003870a94c7e708dc6b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:45.984803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:46.015014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1050/2087] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:45 2022] POST /identity/v3/projects => generated 354 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:46.018467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:46.092180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec8ac-dbbd-41a9-b046-566ba71b8928 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:46.103414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66421b04cccf435f909d0528ce4016ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8242a850b34a4532a8de58d321843a40', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:46.104802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:46.104802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:46.104802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:46.105875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:46.106328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:46.109568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:46.136616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1051/2088] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:50:46.148350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7e7a66a67e4003870a94c7e708dc6b', 'request_id': 'req-215cce06-0294-494c-be77-daaa34b7964d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.155778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7e7a66a67e4003870a94c7e708dc6b', 'request_id': 'req-215cce06-0294-494c-be77-daaa34b7964d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.169594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7e7a66a67e4003870a94c7e708dc6b', 'request_id': 'req-215cce06-0294-494c-be77-daaa34b7964d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.206464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:46.220153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2dc9e-e24d-4185-9485-720bec66c208 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:46.292114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66421b04cccf435f909d0528ce4016ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8242a850b34a4532a8de58d321843a40', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:46.293031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:46.293328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:46.293670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] PATH_INFO: `/v3/projects/b91e8104e0a94689b4f0cb5255872921` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:46.294885 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b91e8104e0a94689b4f0cb5255872921)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:46.297340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] The token cache is being invalidate because project da7e7a66a67e4003870a94c7e708dc6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:46.299589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:46.299740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.299740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.300803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1038/2089] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:45 2022] DELETE /identity/v3/projects/da7e7a66a67e4003870a94c7e708dc6b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:46.304590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:46.306275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:46.376267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cce06-0294-494c-be77-daaa34b7964d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:46.394135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:46.394135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:46.394135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:46.394135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] PATH_INFO: `/v3/users/f4501a5c12654e388b4848a443276b7d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:46.394584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4501a5c12654e388b4848a443276b7d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:46.394584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:46.429208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b91e8104e0a94689b4f0cb5255872921', 'request_id': 'req-3b85eb32-237c-41d4-b391-2e9ef36bce08'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.435828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b91e8104e0a94689b4f0cb5255872921', 'request_id': 'req-3b85eb32-237c-41d4-b391-2e9ef36bce08'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.519746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b91e8104e0a94689b4f0cb5255872921', 'request_id': 'req-3b85eb32-237c-41d4-b391-2e9ef36bce08'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.522846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:46.531609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] The token cache is being invalidate because project b91e8104e0a94689b4f0cb5255872921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:46.531903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.532612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.536667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1052/2090] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:46 2022] DELETE /identity/v3/projects/b91e8104e0a94689b4f0cb5255872921 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:46.536667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:46.541206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:46.631914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4501a5c12654e388b4848a443276b7d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.649336 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4501a5c12654e388b4848a443276b7d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.650006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4501a5c12654e388b4848a443276b7d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.650624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4501a5c12654e388b4848a443276b7d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.659319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4501a5c12654e388b4848a443276b7d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.660199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85eb32-237c-41d4-b391-2e9ef36bce08 tempest-VolumeQuotaClassesTest-1500642825 tempest-VolumeQuotaClassesTest-1500642825-project-admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:46.668950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:46.669935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:46.670223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:46.670559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] PATH_INFO: `/v3/projects/d2cb127338f440628d8c250c16c1c047` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:46.672163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2cb127338f440628d8c250c16c1c047)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:46.675676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:46.770081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1039/2091] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:46 2022] DELETE /identity/v3/users/f4501a5c12654e388b4848a443276b7d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:46.818526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2cb127338f440628d8c250c16c1c047', 'request_id': 'req-64ad4e73-3d21-449b-aea8-c1f3502bb577'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.825513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2cb127338f440628d8c250c16c1c047', 'request_id': 'req-64ad4e73-3d21-449b-aea8-c1f3502bb577'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.850565 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:46.936822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2cb127338f440628d8c250c16c1c047', 'request_id': 'req-64ad4e73-3d21-449b-aea8-c1f3502bb577'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.947641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] The token cache is being invalidate because project d2cb127338f440628d8c250c16c1c047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:46.947930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.948600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:46.950156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1053/2092] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:46 2022] DELETE /identity/v3/projects/d2cb127338f440628d8c250c16c1c047 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:46.954329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:46.955767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:47.039056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68270ac0-5576-41d6-aa09-a959375b3025 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:47.146111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:47.147369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:47.147369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:47.147605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] PATH_INFO: `/v3/users/a66a3ba723434c589dfd63e3d5fc11e8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:47.149213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66a3ba723434c589dfd63e3d5fc11e8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:47.151588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:47.337323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:47.339153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:47.358417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e73-3d21-449b-aea8-c1f3502bb577 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:47.364659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:47.365547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:47.365826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:47.366100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] PATH_INFO: `/v3/projects/09343de02e174191894a09ce364f9374` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:47.367336 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09343de02e174191894a09ce364f9374)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:47.369768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:47.437037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66a3ba723434c589dfd63e3d5fc11e8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.449177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66a3ba723434c589dfd63e3d5fc11e8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.450194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66a3ba723434c589dfd63e3d5fc11e8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.450954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66a3ba723434c589dfd63e3d5fc11e8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.459183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66a3ba723434c589dfd63e3d5fc11e8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.527568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09343de02e174191894a09ce364f9374', 'request_id': 'req-90d7c076-5d40-487e-ae11-fb76a8e17809'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.533930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09343de02e174191894a09ce364f9374', 'request_id': 'req-90d7c076-5d40-487e-ae11-fb76a8e17809'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.542962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1040/2093] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:46 2022] DELETE /identity/v3/users/a66a3ba723434c589dfd63e3d5fc11e8 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:47.546181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:47.546181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:47.612168 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4e522-5b6b-46e7-bb23-a0cc7ceae138 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:47.616782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:47.617665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:47.617936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:47.618208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] PATH_INFO: `/v3/users/fab4b74ababf44f88748a98ff67c73a2` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:47.619442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fab4b74ababf44f88748a98ff67c73a2)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:47.621426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:47.657683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09343de02e174191894a09ce364f9374', 'request_id': 'req-90d7c076-5d40-487e-ae11-fb76a8e17809'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.671917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] The token cache is being invalidate because project 09343de02e174191894a09ce364f9374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:47.673123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.673969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-90d7c076-5d40-487e-ae11-fb76a8e17809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.676039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1054/2094] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:47 2022] DELETE /identity/v3/projects/09343de02e174191894a09ce364f9374 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:47.682644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faceb12-c3f0-49d7-ab69-9c53cc08221a None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:47.683044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faceb12-c3f0-49d7-ab69-9c53cc08221a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:47.683423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faceb12-c3f0-49d7-ab69-9c53cc08221a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:47.707154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5faceb12-c3f0-49d7-ab69-9c53cc08221a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:47.708596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5faceb12-c3f0-49d7-ab69-9c53cc08221a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:47.725799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1055/2095] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:47.735324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd82ae6-1e7d-4443-95f1-076f82af979a None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:47.735913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd82ae6-1e7d-4443-95f1-076f82af979a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:47.736365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd82ae6-1e7d-4443-95f1-076f82af979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:47.760630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5cd82ae6-1e7d-4443-95f1-076f82af979a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:47.762506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5cd82ae6-1e7d-4443-95f1-076f82af979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:47.781200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1056/2096] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:47.782741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd82ae6-1e7d-4443-95f1-076f82af979a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:47.798618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd82ae6-1e7d-4443-95f1-076f82af979a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:47.803753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:47.804659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:47.805016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:47.805365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:47.806195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:47.806569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:47.808570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:47.832065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1057/2097] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:47.833986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:47.845349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475789-fffc-4b8d-ae52-7c2b4cc6949b admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:47.851096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eefb4-0630-44da-bb6c-a442bdf412ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:47.852014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eefb4-0630-44da-bb6c-a442bdf412ec admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:47.852511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eefb4-0630-44da-bb6c-a442bdf412ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:47.853043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eefb4-0630-44da-bb6c-a442bdf412ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:47.854035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eefb4-0630-44da-bb6c-a442bdf412ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:47.856087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eefb4-0630-44da-bb6c-a442bdf412ec admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:47.894725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fab4b74ababf44f88748a98ff67c73a2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.917031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fab4b74ababf44f88748a98ff67c73a2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.917785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fab4b74ababf44f88748a98ff67c73a2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.918520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fab4b74ababf44f88748a98ff67c73a2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:47.926837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fab4b74ababf44f88748a98ff67c73a2'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:48.028879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1041/2098] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:47 2022] DELETE /identity/v3/users/fab4b74ababf44f88748a98ff67c73a2 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:48.029508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1058/2099] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:47 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:48.033420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.034459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.037383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.037507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.037601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.060567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:48.062639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.119196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052d687-fecb-493c-955b-358c06ef7731 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.164483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1059/2100] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:48.175599 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.176917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.207622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.208530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.208823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.209105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.209869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.212035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.232498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:48.275992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286488-0d26-4e09-bcff-fb023e7b6f26 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.279659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.280356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.280704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.280992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.291342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.292928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.304396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1060/2101] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:48.308016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.308016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.323189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207080-fb78-4d7a-a32e-acd5a87b3731 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.334971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.336216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.336698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.337146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] PATH_INFO: `/v3/projects/8fd8eac40af54075afccaf924f4a87c8` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.338614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd8eac40af54075afccaf924f4a87c8)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.342282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1042/2102] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:48 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:48.342418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.342509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.376979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c60f8-3b33-4459-a60b-cb64a530f0bc admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.381727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.382413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.382705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.382989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.383731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.385634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.396137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1043/2103] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:48.399991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.409692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac356d5-e4bc-4ac2-a6ba-89c7eb4f4f15 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.414754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.415463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.415735 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.416065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.416838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.422607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.429930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1044/2104] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:48.434121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.450056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919e6e3-6dfc-40df-829b-b629492a7a1d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.461022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e3b5e-28e1-4bd2-906d-de7a311aac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.462544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3b5e-28e1-4bd2-906d-de7a311aac7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.462839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3b5e-28e1-4bd2-906d-de7a311aac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.463120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3b5e-28e1-4bd2-906d-de7a311aac7d admin admin] PATH_INFO: `/v3/projects/29f6e63ebf564411b151a52dc2af4f5e/users/4fc27b06dfa848b092c04a5a02180a7f/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.468333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd8eac40af54075afccaf924f4a87c8', 'request_id': 'req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:48.474814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd8eac40af54075afccaf924f4a87c8', 'request_id': 'req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:48.480752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e3b5e-28e1-4bd2-906d-de7a311aac7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f6e63ebf564411b151a52dc2af4f5e, user_id=4fc27b06dfa848b092c04a5a02180a7f, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.486799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e3b5e-28e1-4bd2-906d-de7a311aac7d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.584464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd8eac40af54075afccaf924f4a87c8', 'request_id': 'req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:48.594199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1045/2105] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:48 2022] PUT /identity/v3/projects/29f6e63ebf564411b151a52dc2af4f5e/users/4fc27b06dfa848b092c04a5a02180a7f/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:48.610167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.611219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.611803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.616571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] The token cache is being invalidate because project 8fd8eac40af54075afccaf924f4a87c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:48.617364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:48.617939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1672a6d9-b723-4bb6-8aba-fa1eea2fc939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:48.619540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1061/2106] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:48 2022] DELETE /identity/v3/projects/8fd8eac40af54075afccaf924f4a87c8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:48.625126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c646d5-9fe1-469c-aa84-76a881756ff6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.625426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c646d5-9fe1-469c-aa84-76a881756ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.625762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c646d5-9fe1-469c-aa84-76a881756ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.662783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] MFA Rules not processed for user `4fc27b06dfa848b092c04a5a02180a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:48.664088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.664219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b5c646d5-9fe1-469c-aa84-76a881756ff6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:48.665375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b5c646d5-9fe1-469c-aa84-76a881756ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.669153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:48.775557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1062/2107] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:48.778433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d4f53-a97d-44b8-b2cf-d8024910a57c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.778936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d4f53-a97d-44b8-b2cf-d8024910a57c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.779346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d4f53-a97d-44b8-b2cf-d8024910a57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.780775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1046/2108] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:48.792024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.796598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.812007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0f6d4f53-a97d-44b8-b2cf-d8024910a57c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:48.813704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0f6d4f53-a97d-44b8-b2cf-d8024910a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.827340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1063/2109] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:48.831309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d4f53-a97d-44b8-b2cf-d8024910a57c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.842627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d4f53-a97d-44b8-b2cf-d8024910a57c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.848178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.848897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.849182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.849458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.850198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:48.850513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.852169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.880674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1064/2110] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:50:48.881723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.892955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bf040-389f-4089-8b25-f02025493d2e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.894013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b11fe-4cd7-41da-b1b2-d5fc59b3dfeb None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.898325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.899066 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.902069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.902069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.902256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.902256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.902256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.902948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.903874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.908484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.911121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.914404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:48.932911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1047/2111] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:48.940362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:48.951867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:48.968842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0512e3c-6307-4aac-82b6-d1ef1fe17011 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:48.976721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:48.977902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:48.978136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:48.978518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] PATH_INFO: `/v3/projects/170e06a5f54b4570a24b230c7eefcb59` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:48.979879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] RBAC: Authorizing `identity:delete_project(project_id=170e06a5f54b4570a24b230c7eefcb59)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:48.982006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:49.060324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1065/2112] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:48 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:49.061548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:49.064679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:49.137512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc56313-34ab-4de6-9978-82db37da228f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:49.143924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:49.145002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:49.145288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:49.145752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] PATH_INFO: `/v3/users/4fc27b06dfa848b092c04a5a02180a7f` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:49.147099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc27b06dfa848b092c04a5a02180a7f)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:49.149065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:49.199103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170e06a5f54b4570a24b230c7eefcb59', 'request_id': 'req-aba3e445-3948-4fe9-8e53-9ea234b3133a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.212217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170e06a5f54b4570a24b230c7eefcb59', 'request_id': 'req-aba3e445-3948-4fe9-8e53-9ea234b3133a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.219997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170e06a5f54b4570a24b230c7eefcb59', 'request_id': 'req-aba3e445-3948-4fe9-8e53-9ea234b3133a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.323858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] The token cache is being invalidate because project 170e06a5f54b4570a24b230c7eefcb59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:49.324187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.324836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.330063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1048/2113] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:48 2022] DELETE /identity/v3/projects/170e06a5f54b4570a24b230c7eefcb59 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:49.331105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:49.332629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:49.446776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc27b06dfa848b092c04a5a02180a7f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.447901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc27b06dfa848b092c04a5a02180a7f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.448385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc27b06dfa848b092c04a5a02180a7f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.462302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc27b06dfa848b092c04a5a02180a7f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.476089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e445-3948-4fe9-8e53-9ea234b3133a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:49.483893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8f54c-bf19-42e9-8d8b-1545c1c57962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:49.485159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f54c-bf19-42e9-8d8b-1545c1c57962 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:49.485159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f54c-bf19-42e9-8d8b-1545c1c57962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:49.485615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f54c-bf19-42e9-8d8b-1545c1c57962 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:49.486028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8f54c-bf19-42e9-8d8b-1545c1c57962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:49.493440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8f54c-bf19-42e9-8d8b-1545c1c57962 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:49.560275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc27b06dfa848b092c04a5a02180a7f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:49.582288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1066/2114] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:49 2022] DELETE /identity/v3/users/4fc27b06dfa848b092c04a5a02180a7f => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:49.583316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:49.585091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:49.598213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1049/2115] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:49 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:49.600890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:49.601413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:49.601927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:49.639820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:49.641769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:49.708815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1c5d-96e8-4053-aeae-13e9e2b9bdfa admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:49.727174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:49.728723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:49.729166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:49.729598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] PATH_INFO: `/v3/users/66421b04cccf435f909d0528ce4016ae` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:49.734328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66421b04cccf435f909d0528ce4016ae)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:49.736669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:49.748362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1050/2116] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:49.748362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:49.755450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:49.782857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83abad-0f3e-4762-b4cd-5fbb8a8ccbf1 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:49.788333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:49.789280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:49.789879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:49.790299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:49.791296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:49.794125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:49.812200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1051/2117] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:49.813329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:49.816133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:49.910033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8f9f9-8314-449b-939e-7fdf89f2e99a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:49.915813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:49.916395 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:49.916883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:49.918098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:49.939308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:49.950412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:49.963867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1052/2118] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:49.965098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:49.977359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573235c-77a2-413d-a774-e7a9699e1809 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:49.986452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:49.991600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:49.991600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:49.991600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:49.991600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:49.991600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:50.019025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1053/2119] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:50.023016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:50.034201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c21c3-5a17-4f6c-88c1-6c0e86aae94a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:50.039352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:50.040627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:50.040989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:50.041238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] PATH_INFO: `/v3/projects/8943d688f5c8489398c9a2e4f789674b/users/bf465a0212c14d43bb6ea57593f92bc0/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:50.050177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66421b04cccf435f909d0528ce4016ae'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:50.051689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66421b04cccf435f909d0528ce4016ae'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:50.052673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66421b04cccf435f909d0528ce4016ae'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:50.066288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66421b04cccf435f909d0528ce4016ae'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:50.078960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8943d688f5c8489398c9a2e4f789674b, user_id=bf465a0212c14d43bb6ea57593f92bc0, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:50.087840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:50.103653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:50.205091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0f9b1db8-35cf-4c36-92a8-ff9a130dbfc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66421b04cccf435f909d0528ce4016ae'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:50.225257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1067/2120] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:49 2022] DELETE /identity/v3/users/66421b04cccf435f909d0528ce4016ae => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:50.241898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1054/2121] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:50 2022] PUT /identity/v3/projects/8943d688f5c8489398c9a2e4f789674b/users/bf465a0212c14d43bb6ea57593f92bc0/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:50.253614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96331c55-b4aa-4a93-96d3-94f66a27f18b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:50.256108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96331c55-b4aa-4a93-96d3-94f66a27f18b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:50.256455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96331c55-b4aa-4a93-96d3-94f66a27f18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:50.297006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-96331c55-b4aa-4a93-96d3-94f66a27f18b None None] MFA Rules not processed for user `bf465a0212c14d43bb6ea57593f92bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:50.298563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-96331c55-b4aa-4a93-96d3-94f66a27f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:50.410983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1068/2122] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:50.415549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:50.417507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:50.480089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:50.513814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d47ee-1be5-4761-84c8-4eb67df092b5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:50.528856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:50.529902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:50.530373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:50.530848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:50.535096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:50.535096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:50.648762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1055/2123] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:50 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:50.653094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96331c55-b4aa-4a93-96d3-94f66a27f18b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:50.660318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:50.666349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:50.672797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96331c55-b4aa-4a93-96d3-94f66a27f18b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:50.752235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ab0b5-f509-4a04-99aa-cb3bf3866650 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:50.757479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:50.758649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:50.759009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:50.759348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] PATH_INFO: `/v3/projects/29f6e63ebf564411b151a52dc2af4f5e` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:50.763661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29f6e63ebf564411b151a52dc2af4f5e)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:50.766384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:50.769259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:50.770569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:50.770915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:50.771270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:50.772082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:50.774855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:50.883025 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1069/2124] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:50 2022] POST /identity/v3/users => generated 299 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:50.888462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:50.912054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296341d-d951-4a53-856e-de5d9cc24e93 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:50.934163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29f6e63ebf564411b151a52dc2af4f5e', 'request_id': 'req-9114389d-f873-4450-98ad-b8056ad95af3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:50.942824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29f6e63ebf564411b151a52dc2af4f5e', 'request_id': 'req-9114389d-f873-4450-98ad-b8056ad95af3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:50.953722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29f6e63ebf564411b151a52dc2af4f5e', 'request_id': 'req-9114389d-f873-4450-98ad-b8056ad95af3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.009394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.009394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.009394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.009394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.009763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.009763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.019075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1070/2125] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:51.019896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.021568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.036357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21fa2c-e753-455d-8a7d-c9f22e802f61 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.041232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.042110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.042415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.042668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] PATH_INFO: `/v3/projects/8242a850b34a4532a8de58d321843a40` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.043860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8242a850b34a4532a8de58d321843a40)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.046507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.063154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] The token cache is being invalidate because project 29f6e63ebf564411b151a52dc2af4f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:51.063573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.064295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.069786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1056/2126] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:50 2022] DELETE /identity/v3/projects/29f6e63ebf564411b151a52dc2af4f5e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:51.070823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.072480 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.085019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114389d-f873-4450-98ad-b8056ad95af3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.095832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0df258-13a5-4dd4-a83a-d46931884e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.096924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0df258-13a5-4dd4-a83a-d46931884e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.097273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0df258-13a5-4dd4-a83a-d46931884e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.097634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0df258-13a5-4dd4-a83a-d46931884e2f admin admin] PATH_INFO: `/v3/projects/398c8215b7084c529a9f5fb07fdc275f/users/5a3a7baaff164e488f5eabf2b5132ca9/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.114947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0df258-13a5-4dd4-a83a-d46931884e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=398c8215b7084c529a9f5fb07fdc275f, user_id=5a3a7baaff164e488f5eabf2b5132ca9, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.117695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0df258-13a5-4dd4-a83a-d46931884e2f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.188597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1057/2127] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:51 2022] PUT /identity/v3/projects/398c8215b7084c529a9f5fb07fdc275f/users/5a3a7baaff164e488f5eabf2b5132ca9/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:51.191538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e48d4-1ece-4aa1-81fb-4a2109fef35d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.191538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e48d4-1ece-4aa1-81fb-4a2109fef35d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.193130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e48d4-1ece-4aa1-81fb-4a2109fef35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.209282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8242a850b34a4532a8de58d321843a40', 'request_id': 'req-58b68885-9095-4cc6-adea-e7b79120d5a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.228971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7f7e48d4-1ece-4aa1-81fb-4a2109fef35d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:51.230238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7f7e48d4-1ece-4aa1-81fb-4a2109fef35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.231076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8242a850b34a4532a8de58d321843a40', 'request_id': 'req-58b68885-9095-4cc6-adea-e7b79120d5a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.319826 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1058/2128] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:51.320470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e48d4-1ece-4aa1-81fb-4a2109fef35d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.324124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8242a850b34a4532a8de58d321843a40', 'request_id': 'req-58b68885-9095-4cc6-adea-e7b79120d5a9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.347190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e48d4-1ece-4aa1-81fb-4a2109fef35d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.347982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] The token cache is being invalidate because project 8242a850b34a4532a8de58d321843a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:51.350401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.350401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-58b68885-9095-4cc6-adea-e7b79120d5a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:51.351178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1071/2129] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:51 2022] DELETE /identity/v3/projects/8242a850b34a4532a8de58d321843a40 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:51.353039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd2fa2-4a78-4f7d-a7ae-0f7480bfb8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.353785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd2fa2-4a78-4f7d-a7ae-0f7480bfb8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.354129 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd2fa2-4a78-4f7d-a7ae-0f7480bfb8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.354886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd2fa2-4a78-4f7d-a7ae-0f7480bfb8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.354996 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.355359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd2fa2-4a78-4f7d-a7ae-0f7480bfb8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.355746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.356399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.357261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd2fa2-4a78-4f7d-a7ae-0f7480bfb8ce admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.370982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1059/2130] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:51.370982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c946c-47a0-472c-9590-3ec925b50aca None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.370982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c946c-47a0-472c-9590-3ec925b50aca None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.370982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c946c-47a0-472c-9590-3ec925b50aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.387698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:51.389166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.398679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1072/2131] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:51.399457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.403026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.414633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-938c946c-47a0-472c-9590-3ec925b50aca None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:51.415831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89a19-9bda-4530-8fd8-7ebb4d1986e0 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.418214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-938c946c-47a0-472c-9590-3ec925b50aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.421541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf316-5003-443e-aaeb-15f26f8a2316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.422555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf316-5003-443e-aaeb-15f26f8a2316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.422946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf316-5003-443e-aaeb-15f26f8a2316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.423443 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf316-5003-443e-aaeb-15f26f8a2316 admin admin] PATH_INFO: `/v3/projects/398c8215b7084c529a9f5fb07fdc275f/users/5a3a7baaff164e488f5eabf2b5132ca9/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.426144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdf316-5003-443e-aaeb-15f26f8a2316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=398c8215b7084c529a9f5fb07fdc275f, user_id=5a3a7baaff164e488f5eabf2b5132ca9, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.429221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdf316-5003-443e-aaeb-15f26f8a2316 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.429833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1060/2132] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:51.430976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c946c-47a0-472c-9590-3ec925b50aca None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.443058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c946c-47a0-472c-9590-3ec925b50aca None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.465620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1073/2133] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:51 2022] PUT /identity/v3/projects/398c8215b7084c529a9f5fb07fdc275f/users/5a3a7baaff164e488f5eabf2b5132ca9/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:51.470093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd1bda-f96c-466c-8edb-f8e79114d520 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.470724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd1bda-f96c-466c-8edb-f8e79114d520 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.471740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd1bda-f96c-466c-8edb-f8e79114d520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.500493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e4cd1bda-f96c-466c-8edb-f8e79114d520 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:51.502450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e4cd1bda-f96c-466c-8edb-f8e79114d520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.561897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:51.565424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b863ee-220d-4789-9b64-8deddfad34b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.568782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b863ee-220d-4789-9b64-8deddfad34b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.569081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b863ee-220d-4789-9b64-8deddfad34b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.569365 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b863ee-220d-4789-9b64-8deddfad34b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.570150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b863ee-220d-4789-9b64-8deddfad34b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:51.570693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b863ee-220d-4789-9b64-8deddfad34b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.572627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b863ee-220d-4789-9b64-8deddfad34b9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.595257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1061/2134] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:51.603558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eff6fa-54be-4d34-869f-3ae054a6e25c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.603558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eff6fa-54be-4d34-869f-3ae054a6e25c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.603558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eff6fa-54be-4d34-869f-3ae054a6e25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.613231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1074/2135] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:51.613231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd1bda-f96c-466c-8edb-f8e79114d520 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.628299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd1bda-f96c-466c-8edb-f8e79114d520 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.628556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d4eff6fa-54be-4d34-869f-3ae054a6e25c None None] MFA Rules not processed for user `5a3a7baaff164e488f5eabf2b5132ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:51.629841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d4eff6fa-54be-4d34-869f-3ae054a6e25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.640459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.641975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.641975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.641975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.642711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.644933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.727127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1062/2136] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:51.728172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eff6fa-54be-4d34-869f-3ae054a6e25c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.761676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eff6fa-54be-4d34-869f-3ae054a6e25c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.768678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.769745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.770569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.771094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.772147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:51.773266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.777745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.803399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1063/2137] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:51.804652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.804816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1075/2138] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:51 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:51.805816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:51.807134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:51.927665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26c7f5-e6d8-4f20-b8df-b1ac83d855d5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.953288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b359b-dd97-4b0d-a797-eeb954bcd370 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:51.971234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.972102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.972205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.972510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:51.973363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:51.975165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:51.997604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:51.998284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:51.998675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:51.998947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.004377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:52.046388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:52.082544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.082838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1076/2139] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:51 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:52.083761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.086516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.106472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52014a-d054-4d3a-ad28-a586372492b1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.111818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1064/2140] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:52.112706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.117728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.120309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.120620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.120909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.121626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.127424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.138044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39e753-bfc7-4b5f-9660-8e7a01a4b23c service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.148475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.149169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.150584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.151015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.155411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.157811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.168813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1065/2141] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:52.173502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.184215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7e3eb-cf3b-404c-9d23-a411813ff9a5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.189611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.190277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.190604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.190857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.191582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.193366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.204285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1066/2142] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:52.208215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.218572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a9f26-cb82-4bf8-b6de-78bb5296687b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.224100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c842bd-efd2-4881-9483-982a2527a942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.225106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c842bd-efd2-4881-9483-982a2527a942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.225920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c842bd-efd2-4881-9483-982a2527a942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.226215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c842bd-efd2-4881-9483-982a2527a942 admin admin] PATH_INFO: `/v3/projects/fa61d8cf6fe24c418558795c90721386/users/8a39e76ae3f14d0d908effe13ccba911/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.244383 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c842bd-efd2-4881-9483-982a2527a942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa61d8cf6fe24c418558795c90721386, user_id=8a39e76ae3f14d0d908effe13ccba911, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.247192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c842bd-efd2-4881-9483-982a2527a942 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.269302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1077/2143] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:52 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:52.274384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.365324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1067/2144] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:52 2022] PUT /identity/v3/projects/fa61d8cf6fe24c418558795c90721386/users/8a39e76ae3f14d0d908effe13ccba911/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:52.366961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7176c-0a36-45bb-a065-42be307a9059 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.367149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7176c-0a36-45bb-a065-42be307a9059 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.367430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7176c-0a36-45bb-a065-42be307a9059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.368750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf8769-70b6-461d-afc8-5f75ef5a1732 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.421099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-80e7176c-0a36-45bb-a065-42be307a9059 None None] MFA Rules not processed for user `8a39e76ae3f14d0d908effe13ccba911`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:52.422356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-80e7176c-0a36-45bb-a065-42be307a9059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:52.519294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.521239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.521538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.521825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.525374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.529548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.594583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1068/2145] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:52.614208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7176c-0a36-45bb-a065-42be307a9059 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.679533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1078/2146] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:52 2022] POST /identity/v3/users => generated 283 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:52.689427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.694350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7176c-0a36-45bb-a065-42be307a9059 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.700196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.708337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.708646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.709544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.709844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7d09a-24d6-46c4-b9a3-eb258d5d6d1c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.717105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.718115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.718115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.718365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.719092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.719657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.720724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.721277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.732486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1079/2147] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:50:52.732623 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1069/2148] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:52.741662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.755271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b20b5-526a-40e4-8606-772f807b8988 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.767388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22188050-c6c8-433a-b851-bdf2a5ba5351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.768238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188050-c6c8-433a-b851-bdf2a5ba5351 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.768930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188050-c6c8-433a-b851-bdf2a5ba5351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.769220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188050-c6c8-433a-b851-bdf2a5ba5351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.770029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188050-c6c8-433a-b851-bdf2a5ba5351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.771700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188050-c6c8-433a-b851-bdf2a5ba5351 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:52.805856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:52.806644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1080/2149] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:52.832012 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5847e-42be-4bd8-a990-a0c16eb7c738 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:52.837606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71038af7-0a88-4a24-bd17-554beebd21a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:52.839207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71038af7-0a88-4a24-bd17-554beebd21a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:52.843510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71038af7-0a88-4a24-bd17-554beebd21a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:52.843510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71038af7-0a88-4a24-bd17-554beebd21a6 admin admin] PATH_INFO: `/v3/projects/be8f1e48663a41c1b03a8549f0c9ea9a/users/482e79643fa24569b78ccb5b5839c373/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:52.875380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71038af7-0a88-4a24-bd17-554beebd21a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8f1e48663a41c1b03a8549f0c9ea9a, user_id=482e79643fa24569b78ccb5b5839c373, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:52.882948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71038af7-0a88-4a24-bd17-554beebd21a6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:53.021163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1070/2150] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:52 2022] PUT /identity/v3/projects/be8f1e48663a41c1b03a8549f0c9ea9a/users/482e79643fa24569b78ccb5b5839c373/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:53.026450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77703933-45ed-4551-ab44-e2e688f4ac7c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:53.030684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77703933-45ed-4551-ab44-e2e688f4ac7c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:53.030684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77703933-45ed-4551-ab44-e2e688f4ac7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:53.031013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:53.128030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-77703933-45ed-4551-ab44-e2e688f4ac7c None None] MFA Rules not processed for user `482e79643fa24569b78ccb5b5839c373`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:53.129887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-77703933-45ed-4551-ab44-e2e688f4ac7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:53.191132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa0a1-82c8-40b0-99d1-74b2bd667cb2 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:53.191482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa0a1-82c8-40b0-99d1-74b2bd667cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:53.191953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa0a1-82c8-40b0-99d1-74b2bd667cb2 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:53.193828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1071/2151] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:50:53.205457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:53.205793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:53.206074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:53.260479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] MFA Rules not processed for user `4871b13f87f1498ba33f8ecd2a6c0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:53.264250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:53.297668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1081/2152] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:53.303234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77703933-45ed-4551-ab44-e2e688f4ac7c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:53.401855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:53.412199 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1072/2153] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:50:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:50:53.413337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:53.418822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:53.449126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77703933-45ed-4551-ab44-e2e688f4ac7c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:53.454078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:53.454848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:53.455339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:53.455627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:53.456353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:53.458383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:53.491427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98771e-05c3-48bf-ae35-0f8fbf5ff0f7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:53.496657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:53.497808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:53.498091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:53.498414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] PATH_INFO: `/v3/users/137e21587db048529f771d750da5b208` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:53.499783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=137e21587db048529f771d750da5b208)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:53.502002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:53.547161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1082/2154] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:53 2022] POST /identity/v3/projects => generated 365 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:53.547161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:53.559636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:53.630050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cff051-744c-4774-b161-e279f843d176 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:53.635714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:53.636370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:53.636699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:53.636979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:53.647575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:53.649310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:53.659176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1083/2155] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:53.659962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:53.783131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e23ca-574d-4e7b-a966-9f61837f73e6 service placement] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:53.783131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:53.783131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:53.783131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:53.797650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:53.797650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:53.797650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:53.973862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1084/2156] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:53 2022] POST /identity/v3/users => generated 289 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:53.977837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:53.985697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '137e21587db048529f771d750da5b208'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:53.989629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32b776-e279-40b8-8bcf-a2f5fd61228b admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:53.996832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.002106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.002106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.002106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.002106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.002461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '137e21587db048529f771d750da5b208'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.002461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '137e21587db048529f771d750da5b208'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.002613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.002690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '137e21587db048529f771d750da5b208'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.007938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '137e21587db048529f771d750da5b208'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.011855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1085/2157] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:54.016682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.018353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.031145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1953-89f5-4dc4-9c90-25212d12ffda admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.035800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.036639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.036921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.037196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] PATH_INFO: `/v3/projects/46773c9a4eee463189f25407d441f322/users/0bf03a88dbfe49a69dc404459cd54e6c/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.052180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=46773c9a4eee463189f25407d441f322, user_id=0bf03a88dbfe49a69dc404459cd54e6c, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.055003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.117313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1073/2158] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:50:53 2022] DELETE /identity/v3/users/137e21587db048529f771d750da5b208 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:54.158189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1086/2159] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:54 2022] PUT /identity/v3/projects/46773c9a4eee463189f25407d441f322/users/0bf03a88dbfe49a69dc404459cd54e6c/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:50:54.160183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.236031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cab0b-317a-4f06-8abb-a9d75faf0ddb admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.278263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:54.303396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fda34c-99d4-4c62-81d0-facf626e2bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.304660 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fda34c-99d4-4c62-81d0-facf626e2bff admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.304948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fda34c-99d4-4c62-81d0-facf626e2bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.305226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fda34c-99d4-4c62-81d0-facf626e2bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.305998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fda34c-99d4-4c62-81d0-facf626e2bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.307603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fda34c-99d4-4c62-81d0-facf626e2bff admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.318031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1074/2160] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:54.322346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.333392 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ff61a-7282-400b-a250-7039c68061bb admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.338367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.339270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.339552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.339830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] PATH_INFO: `/v3/projects/46773c9a4eee463189f25407d441f322/users/0bf03a88dbfe49a69dc404459cd54e6c/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.341890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46773c9a4eee463189f25407d441f322, user_id=0bf03a88dbfe49a69dc404459cd54e6c, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.345166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.355642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1087/2161] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:54 2022] PUT /identity/v3/projects/46773c9a4eee463189f25407d441f322/users/0bf03a88dbfe49a69dc404459cd54e6c/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:54.367871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.379481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.379481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.397422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] MFA Rules not processed for user `0bf03a88dbfe49a69dc404459cd54e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:54.398673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.398810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.400403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.474385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1075/2162] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:54.485350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869881-b4a3-40b5-9054-a56dda225111 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.489451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.492268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.492374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.492374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.492585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.492826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] PATH_INFO: `/v3/projects/6a76562649474d5bb8f326aab70630ab` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.494078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a76562649474d5bb8f326aab70630ab)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.495981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.551100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a8056-d7c1-40d6-961b-ecea02bf186f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.554821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58145029-b423-44bd-943b-3c4d8f398fd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.555411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58145029-b423-44bd-943b-3c4d8f398fd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.555694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58145029-b423-44bd-943b-3c4d8f398fd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.555973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58145029-b423-44bd-943b-3c4d8f398fd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.565470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58145029-b423-44bd-943b-3c4d8f398fd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.566912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58145029-b423-44bd-943b-3c4d8f398fd8 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.576379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1076/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:54 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:50:54.607669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a76562649474d5bb8f326aab70630ab', 'request_id': 'req-8d706c87-8d7b-42c1-ac0b-2b68131f4269'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.614147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a76562649474d5bb8f326aab70630ab', 'request_id': 'req-8d706c87-8d7b-42c1-ac0b-2b68131f4269'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.697328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a76562649474d5bb8f326aab70630ab', 'request_id': 'req-8d706c87-8d7b-42c1-ac0b-2b68131f4269'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.708803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] The token cache is being invalidate because project 6a76562649474d5bb8f326aab70630ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:50:54.709092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.709839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:50:54.711363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1088/2164] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:50:54 2022] DELETE /identity/v3/projects/6a76562649474d5bb8f326aab70630ab => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:54.716670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9f5fa-7c8d-43f8-a9e2-e53eb5d773c1 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.716987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9f5fa-7c8d-43f8-a9e2-e53eb5d773c1 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.717265 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9f5fa-7c8d-43f8-a9e2-e53eb5d773c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.735716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-73a9f5fa-7c8d-43f8-a9e2-e53eb5d773c1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:54.736991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-73a9f5fa-7c8d-43f8-a9e2-e53eb5d773c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.796751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1077/2165] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:54.797786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.799231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.810966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9287ff-6b74-4ad2-9932-eebafa275413 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.810966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9287ff-6b74-4ad2-9932-eebafa275413 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.810966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9287ff-6b74-4ad2-9932-eebafa275413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.823550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ab9287ff-6b74-4ad2-9932-eebafa275413 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:54.824874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ab9287ff-6b74-4ad2-9932-eebafa275413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.836578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1078/2166] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:54.841453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9287ff-6b74-4ad2-9932-eebafa275413 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.852965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9287ff-6b74-4ad2-9932-eebafa275413 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.858561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.859236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.859861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.860519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d706c87-8d7b-42c1-ac0b-2b68131f4269 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.860676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.861511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:50:54.861905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.864457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.865236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.866421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.867175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.867507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.869761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:54.877754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1079/2167] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:50:54.881766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:54.891656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87232427-f217-4a36-8ff7-fbd55648626a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:54.896461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:54.897138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:54.897426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:54.897704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:54.898390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.900402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.931819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:54.935566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:54.942844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1089/2168] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:54.994663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:55.004252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1080/2169] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:50:54 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:50:55.008222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:55.021887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e51bcb-7fe3-4dee-97b2-9bc390f8d19b service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:55.107672 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:55.108587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:55.109187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:55.110091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:55.111063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:55.113089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:55.218871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1090/2170] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:50:55 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:50:55.219535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:55.232008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07ba16-45cc-4ba3-8e47-95d23062f048 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:55.239452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:55.239452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:55.239452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:55.239452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:55.239901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:55.241599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:55.281178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1081/2171] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:55.291370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:55.303320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34674-3047-44ba-a06a-235020930bbb admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:55.309519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be429655-58fa-4231-b633-b674eaa77607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:55.310206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be429655-58fa-4231-b633-b674eaa77607 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:55.310496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be429655-58fa-4231-b633-b674eaa77607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:55.310906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be429655-58fa-4231-b633-b674eaa77607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:55.311859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be429655-58fa-4231-b633-b674eaa77607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:55.324886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be429655-58fa-4231-b633-b674eaa77607 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:55.349361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1091/2172] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:50:55.351038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:55.361238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa7ac4-cf26-4119-82ba-8421bceb3805 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:55.366755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:55.367708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:55.368125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:55.368402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] PATH_INFO: `/v3/projects/bde6b55e64864d188ee5fafc4b9edc4a/users/8daac4acc2fb4d47a7549ef710a70fca/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:55.385023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde6b55e64864d188ee5fafc4b9edc4a, user_id=8daac4acc2fb4d47a7549ef710a70fca, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:55.387782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:55.474241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1082/2173] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:50:55 2022] PUT /identity/v3/projects/bde6b55e64864d188ee5fafc4b9edc4a/users/8daac4acc2fb4d47a7549ef710a70fca/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:50:55.479673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f487e8-d999-43a9-aaa7-1268d71d05d2 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:55.479976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f487e8-d999-43a9-aaa7-1268d71d05d2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:55.480261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f487e8-d999-43a9-aaa7-1268d71d05d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:55.496196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:55.501842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-50f487e8-d999-43a9-aaa7-1268d71d05d2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:50:55.503101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-50f487e8-d999-43a9-aaa7-1268d71d05d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:55.563909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1092/2174] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:50:55.573446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:50:55.575207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:50:55.645973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4920599-760d-4d8a-aab3-e42e027abbca admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:50:55.649944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa2023-ce23-495a-9044-25d7f310a2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:50:55.650834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2023-ce23-495a-9044-25d7f310a2ab service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:50:55.651182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2023-ce23-495a-9044-25d7f310a2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:50:55.651735 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2023-ce23-495a-9044-25d7f310a2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:50:55.667552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa2023-ce23-495a-9044-25d7f310a2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:50:55.668065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa2023-ce23-495a-9044-25d7f310a2ab service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:50:55.678751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1083/2175] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:50:56.217048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:56.326109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:57.005548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:57.205873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:57.582590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:57.959010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:58.604421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:58.932274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:50:59.680817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:00.016276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:00.430538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f487e8-d999-43a9-aaa7-1268d71d05d2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:00.498927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f487e8-d999-43a9-aaa7-1268d71d05d2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:00.502610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75824e5b-202f-4432-949d-02eeb8e2d90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:00.503299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75824e5b-202f-4432-949d-02eeb8e2d90c service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:00.503583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75824e5b-202f-4432-949d-02eeb8e2d90c service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:00.503891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75824e5b-202f-4432-949d-02eeb8e2d90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:00.505764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-75824e5b-202f-4432-949d-02eeb8e2d90c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:00.578633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75824e5b-202f-4432-949d-02eeb8e2d90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:00.580750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75824e5b-202f-4432-949d-02eeb8e2d90c service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:00.592394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1093/2176] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:01.399592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:01.605173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:01.719521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:01.719825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:01.720184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:01.752516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] MFA Rules not processed for user `8daac4acc2fb4d47a7549ef710a70fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:01.754355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:01.850931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1084/2177] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:01.864415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e49f8b-375d-4977-8e92-a71f975d5769 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:01.864740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e49f8b-375d-4977-8e92-a71f975d5769 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:01.865021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e49f8b-375d-4977-8e92-a71f975d5769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:01.912689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b8e49f8b-375d-4977-8e92-a71f975d5769 None None] MFA Rules not processed for user `8daac4acc2fb4d47a7549ef710a70fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:01.914229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b8e49f8b-375d-4977-8e92-a71f975d5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:01.930096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1094/2178] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:01.941524 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:01.943134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:02.041696 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691752ad-bfea-4727-8cc9-48b8fe6349c7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:02.047873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:02.048645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:02.048932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:02.049216 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:02.060009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:02.061716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:02.071790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1085/2179] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:02.493070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:03.050726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc81fe-d86f-4565-98c8-233827e0ea4f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:03.051257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc81fe-d86f-4565-98c8-233827e0ea4f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:03.051357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc81fe-d86f-4565-98c8-233827e0ea4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:03.072367 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-53dc81fe-d86f-4565-98c8-233827e0ea4f None None] MFA Rules not processed for user `8daac4acc2fb4d47a7549ef710a70fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:03.074024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-53dc81fe-d86f-4565-98c8-233827e0ea4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:03.085268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1095/2180] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:05.643000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:05.644694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:05.662866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39af169-500c-4387-8d5c-8b0296ca8de4 service nova] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:05.668315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:05.671652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:05.672728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:05.673314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] PATH_INFO: `/v3/users/482e79643fa24569b78ccb5b5839c373` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:05.675312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=482e79643fa24569b78ccb5b5839c373)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:05.677550 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:05.968645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '482e79643fa24569b78ccb5b5839c373'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:05.981037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '482e79643fa24569b78ccb5b5839c373'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:05.981775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '482e79643fa24569b78ccb5b5839c373'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:05.982460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '482e79643fa24569b78ccb5b5839c373'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:05.990654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '482e79643fa24569b78ccb5b5839c373'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:06.093605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1086/2181] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:05 2022] DELETE /identity/v3/users/482e79643fa24569b78ccb5b5839c373 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:06.103627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:06.104358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:06.105373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:06.146806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:06.147922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:06.233221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1096/2182] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:51:06.243618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:06.249139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:06.314158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349337-df77-4bac-8f57-51568a8b33a6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:06.319019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:06.319886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:06.320414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:06.320793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:06.335217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:06.336981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:06.355868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1087/2183] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:07.099877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:07.108756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:07.125283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fcf73-5ea3-4f75-9fad-44c7d7c93976 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:07.131112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:07.132747 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:07.133247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:07.133847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] PATH_INFO: `/v3/projects/be8f1e48663a41c1b03a8549f0c9ea9a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:07.135314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8f1e48663a41c1b03a8549f0c9ea9a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:07.138255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:07.367699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8f1e48663a41c1b03a8549f0c9ea9a', 'request_id': 'req-1bf50863-05f8-4293-b906-fc128319899f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.385527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8f1e48663a41c1b03a8549f0c9ea9a', 'request_id': 'req-1bf50863-05f8-4293-b906-fc128319899f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.502808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8f1e48663a41c1b03a8549f0c9ea9a', 'request_id': 'req-1bf50863-05f8-4293-b906-fc128319899f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.514352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] The token cache is being invalidate because project be8f1e48663a41c1b03a8549f0c9ea9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:07.514659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.515278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.517109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1097/2184] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:07 2022] DELETE /identity/v3/projects/be8f1e48663a41c1b03a8549f0c9ea9a => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:07.521122 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:07.530630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed3bf7-77ce-4b3e-af46-84108463f01f service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:07.586710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:07.587579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:07.587864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:07.588143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] PATH_INFO: `/v3/users/0bf03a88dbfe49a69dc404459cd54e6c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:07.589350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf03a88dbfe49a69dc404459cd54e6c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:07.591271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:07.604309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:07.905473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf03a88dbfe49a69dc404459cd54e6c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.918257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf03a88dbfe49a69dc404459cd54e6c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.919002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf03a88dbfe49a69dc404459cd54e6c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.919680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf03a88dbfe49a69dc404459cd54e6c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:07.927594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8bfabef7-f139-48f5-b8c6-1c8ff19397db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf03a88dbfe49a69dc404459cd54e6c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:08.028201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1088/2185] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:07 2022] DELETE /identity/v3/users/0bf03a88dbfe49a69dc404459cd54e6c => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:08.695875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:08.697945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:08.712423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf50863-05f8-4293-b906-fc128319899f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:08.764455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:08.781736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:08.782683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:08.782998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:08.783387 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] PATH_INFO: `/v3/projects/46773c9a4eee463189f25407d441f322` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:08.784528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46773c9a4eee463189f25407d441f322)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:08.786562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:08.909262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46773c9a4eee463189f25407d441f322', 'request_id': 'req-8103e03f-5106-4ca0-89ab-66a24140a819'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:08.915325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46773c9a4eee463189f25407d441f322', 'request_id': 'req-8103e03f-5106-4ca0-89ab-66a24140a819'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:09.043651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46773c9a4eee463189f25407d441f322', 'request_id': 'req-8103e03f-5106-4ca0-89ab-66a24140a819'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:09.054522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] The token cache is being invalidate because project 46773c9a4eee463189f25407d441f322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:09.054812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:09.055452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8103e03f-5106-4ca0-89ab-66a24140a819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:09.057031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1098/2186] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:08 2022] DELETE /identity/v3/projects/46773c9a4eee463189f25407d441f322 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:09.062255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cfb77-7fc2-4a9e-91d8-89309eea7b33 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.062715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cfb77-7fc2-4a9e-91d8-89309eea7b33 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.062849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cfb77-7fc2-4a9e-91d8-89309eea7b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.082503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c54cfb77-7fc2-4a9e-91d8-89309eea7b33 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:09.083811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c54cfb77-7fc2-4a9e-91d8-89309eea7b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:09.143194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1089/2187] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:09.151120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56468938-cd8c-4cd5-b889-9b249f49d016 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.151413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56468938-cd8c-4cd5-b889-9b249f49d016 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.151697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56468938-cd8c-4cd5-b889-9b249f49d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.172999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-56468938-cd8c-4cd5-b889-9b249f49d016 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:09.174252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-56468938-cd8c-4cd5-b889-9b249f49d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:09.182373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1099/2188] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:09.192736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cfb77-7fc2-4a9e-91d8-89309eea7b33 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:09.202833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cfb77-7fc2-4a9e-91d8-89309eea7b33 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:09.207707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:09.208353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.208670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.208947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.209670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:51:09.210001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:09.211597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:09.222909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1090/2189] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:51:09.227200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56468938-cd8c-4cd5-b889-9b249f49d016 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:09.236401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56468938-cd8c-4cd5-b889-9b249f49d016 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:09.241146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:09.241798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.242088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.242371 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.243074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:09.245029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:09.338015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1100/2190] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:09 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:09.342146 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:09.351932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d10f0d-987c-4b49-ab5a-f0243e0ee692 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:09.407408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:09.408025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.408302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.408612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.409305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:09.411248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:09.483069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1091/2191] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:09 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:09.484283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:09.494888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dc74b-70fd-408d-bc51-62baf58eb9f6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:09.500078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:09.500791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.501077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.501359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.502089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:09.503717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:09.518785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1101/2192] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:09.524669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:09.530345 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1079e-8d81-4e32-bcb4-e59fc6524fe0 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:09.535229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18364d5f-20e0-45e1-b45d-1a1156ca3761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:09.535880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18364d5f-20e0-45e1-b45d-1a1156ca3761 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.536162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18364d5f-20e0-45e1-b45d-1a1156ca3761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.536455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18364d5f-20e0-45e1-b45d-1a1156ca3761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.537180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18364d5f-20e0-45e1-b45d-1a1156ca3761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:09.538768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18364d5f-20e0-45e1-b45d-1a1156ca3761 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:09.541027 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:09.550417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1092/2193] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:09.555216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:09.565842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026cabe-7c5b-4ca2-b880-97005fb7e947 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:09.570588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:09.571454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.571735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.572009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] PATH_INFO: `/v3/projects/7050e5efa9214af5b02f9063d2e9242e/users/eab9615daab34e20900c324272e65279/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.585995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7050e5efa9214af5b02f9063d2e9242e, user_id=eab9615daab34e20900c324272e65279, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:09.588665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:09.661968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1102/2194] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:09 2022] PUT /identity/v3/projects/7050e5efa9214af5b02f9063d2e9242e/users/eab9615daab34e20900c324272e65279/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:09.673401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3baca6-2e41-40b1-b419-d2709c112aca None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.673690 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3baca6-2e41-40b1-b419-d2709c112aca None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.673997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3baca6-2e41-40b1-b419-d2709c112aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.701446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ce3baca6-2e41-40b1-b419-d2709c112aca None None] MFA Rules not processed for user `eab9615daab34e20900c324272e65279`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:09.702688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ce3baca6-2e41-40b1-b419-d2709c112aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:09.764943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1093/2195] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:09.775612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:09.777359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:09.841267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c91e2-a04d-4b08-a8c0-fc9e982f5054 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:09.845057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:09.845733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:09.846024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:09.846303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:09.855291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:09.856780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:09.866743 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1103/2196] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:16.078959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:16.079437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:16.079530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:16.097906 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:16.099051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:16.158431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1094/2197] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:51:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:16.201357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:16.202875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:16.221342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:16.284352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673351-bab2-4dfa-9051-cc4189d000cb service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:16.289374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:16.290017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:16.290303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:16.290577 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:16.299242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:16.306636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:16.309251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1104/2198] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:51:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:51:16.372829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:16.374324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:16.479392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9de18-086d-46dc-8083-a5dd32bc1b4a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:16.486598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:16.487979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:16.488568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:16.488944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] PATH_INFO: `/v3/users/bf465a0212c14d43bb6ea57593f92bc0` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:16.490392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf465a0212c14d43bb6ea57593f92bc0)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:16.491793 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:16.503008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:16.508869 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28603d69-9f22-42ec-b413-69fdeefb71c6 service swift] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:16.514150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:16.515353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:16.515638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:16.516012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] PATH_INFO: `/v3/users/eab9615daab34e20900c324272e65279` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:16.517603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab9615daab34e20900c324272e65279)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:16.519906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:16.721880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:16.734643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf465a0212c14d43bb6ea57593f92bc0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.765406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf465a0212c14d43bb6ea57593f92bc0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.766635 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf465a0212c14d43bb6ea57593f92bc0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.767396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf465a0212c14d43bb6ea57593f92bc0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.780631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-59a12df5-c103-48c7-8110-338d4d1b0478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf465a0212c14d43bb6ea57593f92bc0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.825221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab9615daab34e20900c324272e65279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.825910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab9615daab34e20900c324272e65279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.826580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab9615daab34e20900c324272e65279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.836196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab9615daab34e20900c324272e65279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:16.915525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1095/2199] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:16 2022] DELETE /identity/v3/users/bf465a0212c14d43bb6ea57593f92bc0 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:16.920095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:16.921052 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:16.922035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:16.957331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:16.958625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:16.987780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-34a66c03-a1ea-44f5-ab3f-96108433644e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab9615daab34e20900c324272e65279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:17.016487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1105/2200] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:16 2022] DELETE /identity/v3/users/eab9615daab34e20900c324272e65279 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:17.021648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c3938-0601-4477-93b7-ba6e67fe7bbc None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:17.022120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c3938-0601-4477-93b7-ba6e67fe7bbc None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:17.022543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c3938-0601-4477-93b7-ba6e67fe7bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:17.047714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1096/2201] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:17.058040 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:17.059861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:17.065771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-907c3938-0601-4477-93b7-ba6e67fe7bbc None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:17.067070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-907c3938-0601-4477-93b7-ba6e67fe7bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:17.171242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763320d8-c787-4dc9-ac85-4bbe90c75173 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:17.190088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1106/2202] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:17.200417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c3938-0601-4477-93b7-ba6e67fe7bbc None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:17.259835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3e38f-f148-4c23-b86a-9f26d0e4d655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:17.260603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3e38f-f148-4c23-b86a-9f26d0e4d655 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:17.260958 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3e38f-f148-4c23-b86a-9f26d0e4d655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:17.261314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3e38f-f148-4c23-b86a-9f26d0e4d655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:17.274022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3e38f-f148-4c23-b86a-9f26d0e4d655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:17.275014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c3938-0601-4477-93b7-ba6e67fe7bbc None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:17.278140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:17.282263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:17.282626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3e38f-f148-4c23-b86a-9f26d0e4d655 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:17.283686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:17.284229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:17.284729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:17.294680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1097/2203] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:17.297688 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:17.299404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:17.311659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1107/2204] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:17.509380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:17.509677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:17.509962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:17.528842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:17.530600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:17.594937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1098/2205] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jul 20 17:51:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:17.614806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:17.614806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:17.696403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896a769-2acb-4f1e-8a70-36e6c4084a64 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:17.702173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:17.702828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:17.703115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:17.703214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:17.706521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:17.711838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:17.713612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:17.722697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1108/2206] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:51:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:51:17.951087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:17.952148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:18.005725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f8c1b-622f-4010-ac78-73a6ee11805b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:18.011253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:18.012895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:18.015816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:18.018343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.018630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.018908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] PATH_INFO: `/v3/projects/8943d688f5c8489398c9a2e4f789674b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.020860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8943d688f5c8489398c9a2e4f789674b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:18.022870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:18.031134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4bdd-1bf2-4ef5-a273-815b6c8f664e service swift] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:18.037257 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:18.037548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.037776 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.038049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] PATH_INFO: `/v3/projects/7050e5efa9214af5b02f9063d2e9242e` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.039248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7050e5efa9214af5b02f9063d2e9242e)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:18.041243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:18.205292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7050e5efa9214af5b02f9063d2e9242e', 'request_id': 'req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.207674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8943d688f5c8489398c9a2e4f789674b', 'request_id': 'req-3915e642-89d5-42f2-8156-e129f5419353'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.211426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7050e5efa9214af5b02f9063d2e9242e', 'request_id': 'req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.213887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8943d688f5c8489398c9a2e4f789674b', 'request_id': 'req-3915e642-89d5-42f2-8156-e129f5419353'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.221035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8943d688f5c8489398c9a2e4f789674b', 'request_id': 'req-3915e642-89d5-42f2-8156-e129f5419353'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.331496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7050e5efa9214af5b02f9063d2e9242e', 'request_id': 'req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.342875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] The token cache is being invalidate because project 7050e5efa9214af5b02f9063d2e9242e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:18.343165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.343789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4cfa1ddc-4fb1-4228-8412-0d1b84046f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.345364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1109/2207] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:18 2022] DELETE /identity/v3/projects/7050e5efa9214af5b02f9063d2e9242e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:18.350550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68449913-40d4-484c-946e-9d59cf9e0bd1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.350842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68449913-40d4-484c-946e-9d59cf9e0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.351124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68449913-40d4-484c-946e-9d59cf9e0bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.369472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-68449913-40d4-484c-946e-9d59cf9e0bd1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:18.370668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-68449913-40d4-484c-946e-9d59cf9e0bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:18.430520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1110/2208] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:51:18.438954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a873766-e07c-4a34-9703-3da0d9c985ca None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.439305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a873766-e07c-4a34-9703-3da0d9c985ca None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.439592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a873766-e07c-4a34-9703-3da0d9c985ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.459767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5a873766-e07c-4a34-9703-3da0d9c985ca None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:18.459767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5a873766-e07c-4a34-9703-3da0d9c985ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:18.467534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1111/2209] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:51:18.471586 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a873766-e07c-4a34-9703-3da0d9c985ca None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:18.480655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a873766-e07c-4a34-9703-3da0d9c985ca None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:18.485082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] The token cache is being invalidate because project 8943d688f5c8489398c9a2e4f789674b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:18.485423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.488841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.488950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:18.488950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.488950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.489197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1099/2210] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:17 2022] DELETE /identity/v3/projects/8943d688f5c8489398c9a2e4f789674b => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:18.489295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.489295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:51:18.489295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:18.490933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:18.491514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:18.492861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:18.505912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1112/2211] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:51:18.506071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e642-89d5-42f2-8156-e129f5419353 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:18.506899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:18.509407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:18.513601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:18.514663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.517545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.517545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] PATH_INFO: `/v3/users/5a3a7baaff164e488f5eabf2b5132ca9` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.517545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a3a7baaff164e488f5eabf2b5132ca9)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:18.518607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:18.523620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274ce79-c372-4b01-859b-ea1f19f56ca5 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:18.528882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa8085-368b-4ece-9653-af62b5e3b1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:18.530146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8085-368b-4ece-9653-af62b5e3b1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.530564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8085-368b-4ece-9653-af62b5e3b1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.530856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8085-368b-4ece-9653-af62b5e3b1a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.531550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa8085-368b-4ece-9653-af62b5e3b1a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:18.533554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa8085-368b-4ece-9653-af62b5e3b1a8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:18.635138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1113/2212] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:18 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:18.636606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab5494-bd9b-4612-9396-d9f0fd94816a None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.636925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab5494-bd9b-4612-9396-d9f0fd94816a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.637212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab5494-bd9b-4612-9396-d9f0fd94816a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.654238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-70ab5494-bd9b-4612-9396-d9f0fd94816a None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:18.656077 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-70ab5494-bd9b-4612-9396-d9f0fd94816a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:18.710464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1114/2213] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:51:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:18.711227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab5494-bd9b-4612-9396-d9f0fd94816a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:18.762137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a3a7baaff164e488f5eabf2b5132ca9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.787196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a3a7baaff164e488f5eabf2b5132ca9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.788161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a3a7baaff164e488f5eabf2b5132ca9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.789071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a3a7baaff164e488f5eabf2b5132ca9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.794287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab5494-bd9b-4612-9396-d9f0fd94816a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:18.800310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:18.801602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:18.802992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:18.802992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:18.803188 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a3a7baaff164e488f5eabf2b5132ca9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:18.803474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:18.805981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.024906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:19.030132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1100/2214] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:18 2022] DELETE /identity/v3/users/5a3a7baaff164e488f5eabf2b5132ca9 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:19.031060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.032487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.037170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1115/2215] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:18 2022] POST /identity/v3/users => generated 285 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:19.038269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.040812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.059900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20108a20-667c-4473-ac88-93557b7fc792 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.138608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.140762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.141765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.142114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] PATH_INFO: `/v3/users/6d9891dce2124828b706bef92e7a12ac` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.144577 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d9891dce2124828b706bef92e7a12ac)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:19.147620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.159647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:19.163411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17891d-82d7-4260-954d-392d34df5f75 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.170508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.171292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.171702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.172128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.174907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.242724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:19.244391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.253567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1101/2216] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:51:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:51:19.260403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.262401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.278085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad1906-f673-4ff9-99e5-e4d8586c5ccb service swift] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.286022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.288208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.288208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.288208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.289763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:19.291686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.304467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1102/2217] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:19.306477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.317740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c16afa-495e-4de6-be44-2df6b1081653 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.323577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.324315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.324711 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.325062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.325863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:19.328250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.365903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1103/2218] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:51:19.366825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.381621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d9891dce2124828b706bef92e7a12ac'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.382403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d9891dce2124828b706bef92e7a12ac'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.383222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d9891dce2124828b706bef92e7a12ac'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.386819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab13ba-a0b2-4a1e-8890-3f3208a50b02 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.396814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d9891dce2124828b706bef92e7a12ac'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.407092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140fd14-1d86-4aeb-88ab-0bda1ed13550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.408205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140fd14-1d86-4aeb-88ab-0bda1ed13550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.408618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140fd14-1d86-4aeb-88ab-0bda1ed13550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.408905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140fd14-1d86-4aeb-88ab-0bda1ed13550 admin admin] PATH_INFO: `/v3/projects/5ab9da65d4254e21912edde42a4a9b3b/users/1d0aa1ba95b240c0a1174f50a2c0d074/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.428887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140fd14-1d86-4aeb-88ab-0bda1ed13550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab9da65d4254e21912edde42a4a9b3b, user_id=1d0aa1ba95b240c0a1174f50a2c0d074, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:19.431880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140fd14-1d86-4aeb-88ab-0bda1ed13550 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.509812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f667cccf-0eff-4708-9a87-9809e2d233c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d9891dce2124828b706bef92e7a12ac'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.510870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1104/2219] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:19 2022] PUT /identity/v3/projects/5ab9da65d4254e21912edde42a4a9b3b/users/1d0aa1ba95b240c0a1174f50a2c0d074/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:19.521084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.521680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.522290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.533922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1116/2220] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:19 2022] DELETE /identity/v3/users/6d9891dce2124828b706bef92e7a12ac => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:19.535570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.535871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.536155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.555101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:19.556619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.574402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] MFA Rules not processed for user `1d0aa1ba95b240c0a1174f50a2c0d074`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:19.578141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.623007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1117/2221] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:19.633658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.636916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.653604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbadcd-9b62-4ee7-a107-d4187d83e315 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.666859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.666859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.666859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.666859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] PATH_INFO: `/v3/projects/398c8215b7084c529a9f5fb07fdc275f` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.668647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=398c8215b7084c529a9f5fb07fdc275f)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:19.671705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.678731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1105/2222] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:51:19.681390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.687927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.759797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b93bc-e766-47a6-8db4-01101abeed88 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.767988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.767988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.767988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.767988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.777599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:19.779204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:19.796612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1106/2223] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:19.797925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:19.800420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:19.835401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398c8215b7084c529a9f5fb07fdc275f', 'request_id': 'req-2dfee398-b460-4d89-a722-744f2cbf6e52'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.845476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398c8215b7084c529a9f5fb07fdc275f', 'request_id': 'req-2dfee398-b460-4d89-a722-744f2cbf6e52'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.874641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a097c2-a35d-4c19-8ec9-d3a697974e74 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:19.927184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c2c04-b615-41c7-a097-2056d1460d96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:19.929523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c2c04-b615-41c7-a097-2056d1460d96 service glance] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.929523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c2c04-b615-41c7-a097-2056d1460d96 service glance] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.929523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c2c04-b615-41c7-a097-2056d1460d96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.947984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398c8215b7084c529a9f5fb07fdc275f', 'request_id': 'req-2dfee398-b460-4d89-a722-744f2cbf6e52'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.959321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] The token cache is being invalidate because project 398c8215b7084c529a9f5fb07fdc275f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:19.959707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.960541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2dfee398-b460-4d89-a722-744f2cbf6e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:19.962315 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1118/2224] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:19 2022] DELETE /identity/v3/projects/398c8215b7084c529a9f5fb07fdc275f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:19.963734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:19.964092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:19.964568 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:19.985180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:19.986519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.009167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c2c04-b615-41c7-a097-2056d1460d96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.011775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c2c04-b615-41c7-a097-2056d1460d96 service glance] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.034185 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1107/2225] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:20.035503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045c24-0d65-49a1-97db-fde4741621b1 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.035792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045c24-0d65-49a1-97db-fde4741621b1 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.036068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045c24-0d65-49a1-97db-fde4741621b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.057803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-6a045c24-0d65-49a1-97db-fde4741621b1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:20.059010 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6a045c24-0d65-49a1-97db-fde4741621b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.065773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1119/2226] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jul 20 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:20.091886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.093203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.162494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1108/2227] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:20.171125 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7086a9c-2f31-44ac-a2b6-dbb64e56af7a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.171663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7086a9c-2f31-44ac-a2b6-dbb64e56af7a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.172067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7086a9c-2f31-44ac-a2b6-dbb64e56af7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.182910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:20.202547 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d7086a9c-2f31-44ac-a2b6-dbb64e56af7a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:20.203985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d7086a9c-2f31-44ac-a2b6-dbb64e56af7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.222210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1109/2228] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:20.224101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7086a9c-2f31-44ac-a2b6-dbb64e56af7a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.242493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7086a9c-2f31-44ac-a2b6-dbb64e56af7a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.243584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfe1f7-429c-45e0-afc6-f42e2ae55727 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.252969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.253656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.253944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.254753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.255472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.255596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.255770 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.256032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.256350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:51:20.256810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.258387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.266841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.268759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.279858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1110/2229] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:51:20.280925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1120/2230] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:51:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 17:51:20.281515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.299049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd87f-6657-44f0-b74e-fcc7934b4529 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.307347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.308020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.308293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.308624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.309326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.312343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.361377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.362876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.399810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf7dd8-b313-4f8a-8a8d-7f94b46dd9b5 service swift] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.405940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.407098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.407388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.407669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] PATH_INFO: `/v3/projects/2b7907f113c54585af1d8411b4757897` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.409070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7907f113c54585af1d8411b4757897)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.411249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.437703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1111/2231] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:20.442101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.510810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044abfd-ce7d-4a63-89e6-0bc131aa349d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.517000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.519256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.519592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.519793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.520618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.522531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.544727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7907f113c54585af1d8411b4757897', 'request_id': 'req-e9bc2a79-805c-4563-a625-e3d3248c6705'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:20.553450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7907f113c54585af1d8411b4757897', 'request_id': 'req-e9bc2a79-805c-4563-a625-e3d3248c6705'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:20.618099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7907f113c54585af1d8411b4757897', 'request_id': 'req-e9bc2a79-805c-4563-a625-e3d3248c6705'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:20.628487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1112/2232] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:51:20.632609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] The token cache is being invalidate because project 2b7907f113c54585af1d8411b4757897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:20.632747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.633207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:20.634578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e9bc2a79-805c-4563-a625-e3d3248c6705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:20.636080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:20.637418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1121/2233] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:20 2022] DELETE /identity/v3/projects/2b7907f113c54585af1d8411b4757897 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:20.643150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b53462-d9a8-45ac-b96e-4dcbf6406929 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.643485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b53462-d9a8-45ac-b96e-4dcbf6406929 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.643797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b53462-d9a8-45ac-b96e-4dcbf6406929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.648004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febba6b-42d0-4363-983c-92d7f55cbd85 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.663679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f1b53462-d9a8-45ac-b96e-4dcbf6406929 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:20.664934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f1b53462-d9a8-45ac-b96e-4dcbf6406929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.709349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.710051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.710337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.710612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.712174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.713560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.730026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1113/2234] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:51:20.739966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.744266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.759237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94f300-2a51-460c-b78f-87aa5a34d2a5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.760966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1122/2235] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:51:20.763016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8814c-6cf0-4259-b879-606bb0e0f0fe None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.763551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8814c-6cf0-4259-b879-606bb0e0f0fe None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.764091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8814c-6cf0-4259-b879-606bb0e0f0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.767996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.769598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.769751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.769945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.771469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.773114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.788943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1114/2236] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:51:20.791968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.792949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-98f8814c-6cf0-4259-b879-606bb0e0f0fe None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:20.794314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-98f8814c-6cf0-4259-b879-606bb0e0f0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:20.808106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6ae50-5f77-49f0-8071-daf79c8df509 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.815248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88726cd-8b13-4535-a23c-ac1c30cffcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.816204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88726cd-8b13-4535-a23c-ac1c30cffcfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.816523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88726cd-8b13-4535-a23c-ac1c30cffcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.816814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88726cd-8b13-4535-a23c-ac1c30cffcfa admin admin] PATH_INFO: `/v3/projects/468000e93368409a81db454fa5466489/users/7cd3cfbc9f8645179034e9379d982e76/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.830799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1123/2237] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:20.832331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8814c-6cf0-4259-b879-606bb0e0f0fe None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.840943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88726cd-8b13-4535-a23c-ac1c30cffcfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=468000e93368409a81db454fa5466489, user_id=7cd3cfbc9f8645179034e9379d982e76, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.845376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88726cd-8b13-4535-a23c-ac1c30cffcfa admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.852500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8814c-6cf0-4259-b879-606bb0e0f0fe None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.867656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.871433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.871433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.871433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.871433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:51:20.871433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.874306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.890517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1124/2238] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:51:20.900532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:20.921046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73026a4e-bd1d-4e2b-9069-39ba234c4737 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:20.937219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:20.938006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.938305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.938590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:20.939299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:20.946651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:20.950351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1115/2239] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:20 2022] PUT /identity/v3/projects/468000e93368409a81db454fa5466489/users/7cd3cfbc9f8645179034e9379d982e76/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:20.965266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:20.965558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:20.965844 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.020501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1125/2240] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:21.024746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:21.048769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] MFA Rules not processed for user `7cd3cfbc9f8645179034e9379d982e76`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:21.053946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:21.151387 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9280b1-ddf4-44d8-95ba-d93e4d57fe59 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:21.157019 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:21.157761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:21.158045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:21.158325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.159081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:21.161347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:21.191255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1116/2241] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:21.204374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:21.205752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:21.269812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1195c2-edfa-4e2a-89c5-83a734383cb6 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:21.274908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:21.275744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:21.276038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:21.276313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.293249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1126/2242] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:21 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:21.297749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:21.297749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:21.298890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:21.305572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1117/2243] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:51:21.325065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff62f20-bb7a-4e76-82b5-629b870db812 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:21.331534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:21.333508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:21.334848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:21.335335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:21.335879 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.336871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:21.339821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:21.372080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:21.378051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1127/2244] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:21.385548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2adec6-9ef0-4ca8-9efa-2d6fe13c63c5 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:21.404148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e6913-e257-43bd-80ed-3870f3f00642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:21.410799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e6913-e257-43bd-80ed-3870f3f00642 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:21.410799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e6913-e257-43bd-80ed-3870f3f00642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:21.410799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e6913-e257-43bd-80ed-3870f3f00642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.410799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e6913-e257-43bd-80ed-3870f3f00642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:21.410799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e6913-e257-43bd-80ed-3870f3f00642 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:21.421574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1118/2245] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:51:21.426830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:21.439152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d914467-b30b-4303-a31f-216354363a49 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:21.445356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:21.448120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:21.448574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:21.448861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] PATH_INFO: `/v3/projects/63d0c5b02155444fb44a732c46674b8f/users/e72199904d9642e6bf9cacb7af568ddb/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.464705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d0c5b02155444fb44a732c46674b8f, user_id=e72199904d9642e6bf9cacb7af568ddb, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:21.467981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:21.565522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1128/2246] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:21 2022] PUT /identity/v3/projects/63d0c5b02155444fb44a732c46674b8f/users/e72199904d9642e6bf9cacb7af568ddb/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:21.567033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:21.567502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:21.569077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.587416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:21.589035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:21.657938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1119/2247] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:21.667980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:21.669774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:21.762569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02def3e6-ebe5-494f-a0a6-55a4f6e41320 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:21.774808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae555ac-bed1-4216-9f1e-f6fdb1e85310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:21.776767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae555ac-bed1-4216-9f1e-f6fdb1e85310 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:21.777009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae555ac-bed1-4216-9f1e-f6fdb1e85310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:21.777292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae555ac-bed1-4216-9f1e-f6fdb1e85310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:21.800725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:21.817999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae555ac-bed1-4216-9f1e-f6fdb1e85310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:21.820045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae555ac-bed1-4216-9f1e-f6fdb1e85310 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:21.840691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1129/2248] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:22.036011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:22.038567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:22.054128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e5847-903c-4632-8f43-99ef08ec599b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:22.065808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:22.070023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:22.070438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:22.070838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] PATH_INFO: `/v3/users/8a39e76ae3f14d0d908effe13ccba911` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:22.072335 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a39e76ae3f14d0d908effe13ccba911)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:22.074445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:22.218761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:22.343706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a39e76ae3f14d0d908effe13ccba911'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:22.355936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a39e76ae3f14d0d908effe13ccba911'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:22.356760 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a39e76ae3f14d0d908effe13ccba911'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:22.357329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a39e76ae3f14d0d908effe13ccba911'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:22.364523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a39e76ae3f14d0d908effe13ccba911'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:22.425045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1120/2249] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:22 2022] DELETE /identity/v3/users/8a39e76ae3f14d0d908effe13ccba911 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:22.429961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:22.430442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:22.430555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:22.454031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:22.455545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:22.524790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1130/2250] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:22.535552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:22.537851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:22.597227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d470d2-9c2e-4bb7-b4de-80cad69b34ff admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:22.600945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:22.601679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:22.602074 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:22.602361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:22.611759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:22.613290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:22.623290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1121/2251] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:22.731192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:23.006651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:23.332396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:23.334102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:23.353398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6aa91-4ae4-454f-911b-d0416779968d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:23.359790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:23.360717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:23.361009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:23.361297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] PATH_INFO: `/v3/projects/fa61d8cf6fe24c418558795c90721386` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:23.362734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa61d8cf6fe24c418558795c90721386)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:23.365558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:23.487546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:23.490512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:23.508914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c61b0-2a9b-4fd8-8e98-799a14c04ce9 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:23.521514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:23.522786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:23.523308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:23.523753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] PATH_INFO: `/v3/users/694570bd24c74fd19e80bd822048067b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:23.525350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=694570bd24c74fd19e80bd822048067b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:23.527515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:23.533614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa61d8cf6fe24c418558795c90721386', 'request_id': 'req-1ce0119e-9af3-4062-b201-c6cff7417d60'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.549168 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa61d8cf6fe24c418558795c90721386', 'request_id': 'req-1ce0119e-9af3-4062-b201-c6cff7417d60'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.626872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa61d8cf6fe24c418558795c90721386', 'request_id': 'req-1ce0119e-9af3-4062-b201-c6cff7417d60'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.648920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] The token cache is being invalidate because project fa61d8cf6fe24c418558795c90721386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:23.649546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.650911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1ce0119e-9af3-4062-b201-c6cff7417d60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.652880 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1131/2252] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:23 2022] DELETE /identity/v3/projects/fa61d8cf6fe24c418558795c90721386 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:23.736483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '694570bd24c74fd19e80bd822048067b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.768854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '694570bd24c74fd19e80bd822048067b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.771822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '694570bd24c74fd19e80bd822048067b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.773088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '694570bd24c74fd19e80bd822048067b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.787124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '694570bd24c74fd19e80bd822048067b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:23.873730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1122/2253] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:23 2022] DELETE /identity/v3/users/694570bd24c74fd19e80bd822048067b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:23.877981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:23.878559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:23.879102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:23.936268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:23.940015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:24.051346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1132/2254] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:24.064251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:24.066237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:24.154224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2449-66a7-4cb4-a5fa-81dfb76c5c0f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:24.156695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:24.160187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2e1ea-8d89-45b1-b1e3-3e56ebe9e6b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:24.161021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2e1ea-8d89-45b1-b1e3-3e56ebe9e6b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:24.161306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2e1ea-8d89-45b1-b1e3-3e56ebe9e6b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:24.161583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2e1ea-8d89-45b1-b1e3-3e56ebe9e6b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:24.171634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2e1ea-8d89-45b1-b1e3-3e56ebe9e6b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:24.174379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2e1ea-8d89-45b1-b1e3-3e56ebe9e6b6 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:24.184473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1123/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:24.399868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:INSERT=6,keystone:DELETE=3,keystone:SELECT=91 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:24.518573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:24.520592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:24.546224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b4f23-e60e-4af1-a820-dc64848a2727 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:24.552110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:24.553842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:24.554584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:24.555159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] PATH_INFO: `/v3/projects/0bcdae79468045d0b5d773d8caf238e6` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:24.557727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcdae79468045d0b5d773d8caf238e6)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:24.561907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:24.964354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcdae79468045d0b5d773d8caf238e6', 'request_id': 'req-da19c65b-857c-405b-8695-877adb89cafb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:24.971768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcdae79468045d0b5d773d8caf238e6', 'request_id': 'req-da19c65b-857c-405b-8695-877adb89cafb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:25.164635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcdae79468045d0b5d773d8caf238e6', 'request_id': 'req-da19c65b-857c-405b-8695-877adb89cafb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:25.179864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] The token cache is being invalidate because project 0bcdae79468045d0b5d773d8caf238e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:25.179864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:25.182588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-da19c65b-857c-405b-8695-877adb89cafb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:25.185545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1133/2256] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:24 2022] DELETE /identity/v3/projects/0bcdae79468045d0b5d773d8caf238e6 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:25.192182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01967d1a-b6ab-46c6-bdc4-192d535ba7a7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.194847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01967d1a-b6ab-46c6-bdc4-192d535ba7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.195433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01967d1a-b6ab-46c6-bdc4-192d535ba7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.216259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-01967d1a-b6ab-46c6-bdc4-192d535ba7a7 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:25.218139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-01967d1a-b6ab-46c6-bdc4-192d535ba7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:25.290287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1124/2257] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:25.299394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8784f-6d30-4a2c-afb7-3c736a46e761 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.299687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8784f-6d30-4a2c-afb7-3c736a46e761 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.299967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8784f-6d30-4a2c-afb7-3c736a46e761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.319976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e3f8784f-6d30-4a2c-afb7-3c736a46e761 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:25.321243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e3f8784f-6d30-4a2c-afb7-3c736a46e761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:25.330022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1134/2258] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:25.334288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01967d1a-b6ab-46c6-bdc4-192d535ba7a7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:25.344771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01967d1a-b6ab-46c6-bdc4-192d535ba7a7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:25.349776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:25.351368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.351368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.351368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.352563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:51:25.352876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:25.354500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:25.388643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1125/2259] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:51:25.396624 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8784f-6d30-4a2c-afb7-3c736a46e761 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:25.409014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8784f-6d30-4a2c-afb7-3c736a46e761 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:25.414241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:25.414941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.415220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.415643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.416557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:25.418682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:25.532335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1135/2260] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:25 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:25.536723 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:25.553922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02483bd-742f-4d82-89e6-eaa431bb69b0 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:25.628948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:25.630259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.630650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.631156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.632164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:25.634607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:25.779786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1126/2261] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:25 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:25.783977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:25.798654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a50e1b-3059-403b-b3f9-2a38a81d6a2a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:25.803804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:25.804583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.804972 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.805353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.806190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:25.807897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:25.822966 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1136/2262] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:25.824197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:25.836152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516583a-765f-41ab-a730-3f51701ff6ed admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:25.841526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5de23-007f-430d-9ad3-eac0f414d137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:25.842416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5de23-007f-430d-9ad3-eac0f414d137 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.842702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5de23-007f-430d-9ad3-eac0f414d137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.842980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5de23-007f-430d-9ad3-eac0f414d137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.843866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5de23-007f-430d-9ad3-eac0f414d137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:25.845694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5de23-007f-430d-9ad3-eac0f414d137 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:25.862859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1127/2263] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:51:25.872130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:25.881903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ea513-40ec-4a3a-87df-204955312514 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:25.886882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:25.888307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:25.889279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:25.889734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] PATH_INFO: `/v3/projects/10e93d9f177d4ed5bba338ba68547bf7/users/2bc0d03b364946a5b8af3db4e9fd4780/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:25.929194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e93d9f177d4ed5bba338ba68547bf7, user_id=2bc0d03b364946a5b8af3db4e9fd4780, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:25.932226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:26.063081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1137/2264] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:25 2022] PUT /identity/v3/projects/10e93d9f177d4ed5bba338ba68547bf7/users/2bc0d03b364946a5b8af3db4e9fd4780/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:26.073315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf719b-1fd4-4943-ba45-6fc55d9345ac None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:26.073709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf719b-1fd4-4943-ba45-6fc55d9345ac None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:26.074064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf719b-1fd4-4943-ba45-6fc55d9345ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:26.100822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a7bf719b-1fd4-4943-ba45-6fc55d9345ac None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:26.102156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a7bf719b-1fd4-4943-ba45-6fc55d9345ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:26.179816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1128/2265] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:26.190855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:26.192600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:26.278210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b8e4e-39d8-47ac-873e-b0a5d14d39fb admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:26.283015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:26.283766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:26.286492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:26.286492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:26.296845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:26.298605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:26.311338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1138/2266] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:26.411446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:27.006836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:28.836945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5b4ff-ac6d-40ab-97f5-d9b32e0441db None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:28.837548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5b4ff-ac6d-40ab-97f5-d9b32e0441db None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:28.837656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5b4ff-ac6d-40ab-97f5-d9b32e0441db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:28.890110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f7d5b4ff-ac6d-40ab-97f5-d9b32e0441db None None] MFA Rules not processed for user `2bc0d03b364946a5b8af3db4e9fd4780`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:28.890110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f7d5b4ff-ac6d-40ab-97f5-d9b32e0441db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:29.067368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1129/2267] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:29.124101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:29.127614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:29.194802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:29.198837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58ae98-8b3f-451f-986e-12be4ea56cd8 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:29.203573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:29.204199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:29.204514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:29.204794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:29.213503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:29.215053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:29.224155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1139/2268] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:30.295597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6622-ec66-4543-b102-334a3585da27 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:30.296004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6622-ec66-4543-b102-334a3585da27 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:30.296179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6622-ec66-4543-b102-334a3585da27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:30.330705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5aad6622-ec66-4543-b102-334a3585da27 None None] MFA Rules not processed for user `e72199904d9642e6bf9cacb7af568ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:30.332165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5aad6622-ec66-4543-b102-334a3585da27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:30.407999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:30.424476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1130/2269] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:30.424973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:30.430687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:30.448281 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd58fc-b859-4985-8065-77d48b64c854 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:30.456950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:30.457728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:30.458087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:30.458482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:30.462946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:30.462946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:30.562162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1140/2270] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:30 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:51:30.566328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad6622-ec66-4543-b102-334a3585da27 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:30.577758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad6622-ec66-4543-b102-334a3585da27 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:30.637155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:30.638153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:30.638440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:30.638719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:30.639538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:30.641707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:30.746347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1131/2271] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:30 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:51:30.751976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:30.762335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b32bdd-d7b4-462d-8859-3e404d1e024a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:30.786834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:30.787528 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:30.787824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:30.788088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:30.788850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:30.790476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:30.814995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1141/2272] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:30.819250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:30.841100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f035d-74d4-449e-b895-e23acce8b51a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:30.846475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:30.847777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:30.848063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:30.848368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] PATH_INFO: `/v3/projects/5acd24ef057e49868df3d27bfeb3f51d/users/a638868e8f9949549380543e7538160d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:30.875802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acd24ef057e49868df3d27bfeb3f51d, user_id=a638868e8f9949549380543e7538160d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:30.879036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:30.961943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1132/2273] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:30 2022] PUT /identity/v3/projects/5acd24ef057e49868df3d27bfeb3f51d/users/a638868e8f9949549380543e7538160d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:30.966124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:30.980274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e763b-290d-4fc0-8727-87b23e843b07 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:31.074656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9358612-57c3-40fe-ac72-b03a6cb4dc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:31.075353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358612-57c3-40fe-ac72-b03a6cb4dc40 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:31.075643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358612-57c3-40fe-ac72-b03a6cb4dc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:31.075923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358612-57c3-40fe-ac72-b03a6cb4dc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:31.076741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9358612-57c3-40fe-ac72-b03a6cb4dc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:31.082772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9358612-57c3-40fe-ac72-b03a6cb4dc40 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:31.089434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1142/2274] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:31.100158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:31.111808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc968743-fd1e-400f-bc3d-4280ba659f4e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:31.116257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:31.117366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:31.117653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:31.117931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] PATH_INFO: `/v3/projects/5acd24ef057e49868df3d27bfeb3f51d/users/a638868e8f9949549380543e7538160d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:31.119914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acd24ef057e49868df3d27bfeb3f51d, user_id=a638868e8f9949549380543e7538160d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:31.122692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:31.133958 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1133/2275] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:31 2022] PUT /identity/v3/projects/5acd24ef057e49868df3d27bfeb3f51d/users/a638868e8f9949549380543e7538160d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:31.212894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c41c6-bd5b-4383-932f-20689ad9b92f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:31.213234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c41c6-bd5b-4383-932f-20689ad9b92f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:31.213519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c41c6-bd5b-4383-932f-20689ad9b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:31.243921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5e1c41c6-bd5b-4383-932f-20689ad9b92f None None] MFA Rules not processed for user `2bc0d03b364946a5b8af3db4e9fd4780`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:31.245888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5e1c41c6-bd5b-4383-932f-20689ad9b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:31.314978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:31.335035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1143/2276] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:32.855554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:32.858686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:33.009428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98591d50-e120-47cf-85cc-207252cecc6a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:33.058682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:33.059631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:33.060272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:33.061343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] PATH_INFO: `/v3/users/7cd3cfbc9f8645179034e9379d982e76` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:33.066192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd3cfbc9f8645179034e9379d982e76)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:33.068694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:33.345822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd3cfbc9f8645179034e9379d982e76'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:33.348031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:33.367937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd3cfbc9f8645179034e9379d982e76'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:33.368681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd3cfbc9f8645179034e9379d982e76'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:33.369310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd3cfbc9f8645179034e9379d982e76'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:33.380424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd3cfbc9f8645179034e9379d982e76'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:33.465629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1134/2277] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:32 2022] DELETE /identity/v3/users/7cd3cfbc9f8645179034e9379d982e76 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:33.469570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:33.470087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:33.470173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:33.498522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:33.500062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:33.588866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1144/2278] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:33.599491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:33.601236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:33.706131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceca47-5434-4978-96f9-578fbb8729df admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:33.710600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b8ac8-0a6d-4dae-9055-fdeca967713a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:33.711514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b8ac8-0a6d-4dae-9055-fdeca967713a service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:33.711930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b8ac8-0a6d-4dae-9055-fdeca967713a service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:33.712321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b8ac8-0a6d-4dae-9055-fdeca967713a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:33.723683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b8ac8-0a6d-4dae-9055-fdeca967713a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:33.725261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b8ac8-0a6d-4dae-9055-fdeca967713a service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:33.742544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1135/2279] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:34.346179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:34.348350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:34.373465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31b31d-3771-47fb-aca4-a96a27af384b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:34.379302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:34.380258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:34.381962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:34.382735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] PATH_INFO: `/v3/projects/468000e93368409a81db454fa5466489` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:34.384730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] RBAC: Authorizing `identity:delete_project(project_id=468000e93368409a81db454fa5466489)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:34.387273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:34.541917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468000e93368409a81db454fa5466489', 'request_id': 'req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:34.550877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468000e93368409a81db454fa5466489', 'request_id': 'req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:34.662964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468000e93368409a81db454fa5466489', 'request_id': 'req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:34.677837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] The token cache is being invalidate because project 468000e93368409a81db454fa5466489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:34.678159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:34.678782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2d27e7a9-15bd-4d1f-9c2e-3d9574e6137b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:34.681104 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1145/2280] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:34 2022] DELETE /identity/v3/projects/468000e93368409a81db454fa5466489 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:34.686763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834b4c-4b28-47dd-9d80-1a04dddc0df5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:34.687238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834b4c-4b28-47dd-9d80-1a04dddc0df5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:34.687332 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06834b4c-4b28-47dd-9d80-1a04dddc0df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:34.708291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-06834b4c-4b28-47dd-9d80-1a04dddc0df5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:34.710390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-06834b4c-4b28-47dd-9d80-1a04dddc0df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:34.790018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1136/2281] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:34.800345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c692-bbf0-4749-bcfd-37d47e182f54 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:34.800948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c692-bbf0-4749-bcfd-37d47e182f54 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:34.801301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c692-bbf0-4749-bcfd-37d47e182f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:34.824033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4583c692-bbf0-4749-bcfd-37d47e182f54 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:34.825380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4583c692-bbf0-4749-bcfd-37d47e182f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:34.834451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1146/2282] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:34.840243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834b4c-4b28-47dd-9d80-1a04dddc0df5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:34.850022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06834b4c-4b28-47dd-9d80-1a04dddc0df5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:34.855135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:34.855976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:34.856472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:34.856744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:34.857537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:51:34.857848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:34.859491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:34.875535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1137/2283] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:51:34.880258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c692-bbf0-4749-bcfd-37d47e182f54 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:34.891720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c692-bbf0-4749-bcfd-37d47e182f54 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:34.897352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:34.898095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:34.898460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:34.898811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:34.900466 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:34.902748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:35.079071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1147/2284] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:34 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:35.080030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:35.090597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3f7e6-15ed-4d8d-a464-a55069d4ebca admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:35.166081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:35.173842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:35.174919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:35.175387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:35.175956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:35.176972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:35.179247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:35.355015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1138/2285] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:35 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:51:35.359774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:35.386897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4044279-6ee9-4b7e-b80f-eea2227db2b5 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:35.410030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:35.410741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:35.411125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:35.411472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:35.412619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:35.419148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:35.450173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1148/2286] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:35.454375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:35.471072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114d6d1-5d35-4809-bc86-b762f341da67 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:35.481520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d8603-bf38-40c3-949f-fd3538b4e90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:35.484416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d8603-bf38-40c3-949f-fd3538b4e90c admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:35.484822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d8603-bf38-40c3-949f-fd3538b4e90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:35.485179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d8603-bf38-40c3-949f-fd3538b4e90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:35.486011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d8603-bf38-40c3-949f-fd3538b4e90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:35.487774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d8603-bf38-40c3-949f-fd3538b4e90c admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:35.508490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1139/2287] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:35.511493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:35.522629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f64116-3745-49c4-800e-587df572d19f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:35.527878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:35.528813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:35.529097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:35.530690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] PATH_INFO: `/v3/projects/4bbf837501614f9ab0fe56b4cd356663/users/3461ba437a0647b6afcdf1d68bba21b8/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:35.557327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbf837501614f9ab0fe56b4cd356663, user_id=3461ba437a0647b6afcdf1d68bba21b8, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:35.560303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:35.734053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1149/2288] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:35 2022] PUT /identity/v3/projects/4bbf837501614f9ab0fe56b4cd356663/users/3461ba437a0647b6afcdf1d68bba21b8/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:35.736922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4864f-e1cf-47e7-a151-533b2bf6d73b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:35.736922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4864f-e1cf-47e7-a151-533b2bf6d73b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:35.737442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4864f-e1cf-47e7-a151-533b2bf6d73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:35.786005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a9c4864f-e1cf-47e7-a151-533b2bf6d73b None None] MFA Rules not processed for user `3461ba437a0647b6afcdf1d68bba21b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:35.789193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a9c4864f-e1cf-47e7-a151-533b2bf6d73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:36.003888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1140/2289] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:36.010945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:36.060291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:36.182588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab7871-5b22-465b-a2b2-3dc8481dbdc3 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:36.189933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:36.190746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:36.191108 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:36.191537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:36.192251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:36.195238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:36.356764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1150/2290] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:36 2022] POST /identity/v3/projects => generated 385 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:36.362556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4864f-e1cf-47e7-a151-533b2bf6d73b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:36.374182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4864f-e1cf-47e7-a151-533b2bf6d73b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:36.443075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31447a-3717-494d-a254-351058034831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:36.443823 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31447a-3717-494d-a254-351058034831 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:36.444507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31447a-3717-494d-a254-351058034831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:36.444651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31447a-3717-494d-a254-351058034831 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:36.445596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31447a-3717-494d-a254-351058034831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:36.447851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31447a-3717-494d-a254-351058034831 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:36.554516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1141/2291] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:36 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:36.557449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:36.572076 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b771f-d8a3-43c6-bb3d-b5c0fa7ae1b7 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:36.577655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:36.578661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:36.578962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:36.579247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:36.580008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:36.582058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:36.600863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31447a-3717-494d-a254-351058034831 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:36.602089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1151/2292] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:36.613582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31447a-3717-494d-a254-351058034831 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:36.618808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65a27f-1cc9-43a7-a52e-d7a017671057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:36.619501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65a27f-1cc9-43a7-a52e-d7a017671057 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:36.620080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65a27f-1cc9-43a7-a52e-d7a017671057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:36.620375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65a27f-1cc9-43a7-a52e-d7a017671057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:36.621160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65a27f-1cc9-43a7-a52e-d7a017671057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:36.622796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65a27f-1cc9-43a7-a52e-d7a017671057 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:36.636468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1142/2293] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:36.641165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:36.654859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c3b35-c3db-479d-85be-c8eba305ae59 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:36.666706 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:36.668099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:36.668627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:36.669090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] PATH_INFO: `/v3/projects/e880544dafc140f089d4c475f37f4ec0/users/15a6205e2434453db3c007a3c565b2ed/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:36.691048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e880544dafc140f089d4c475f37f4ec0, user_id=15a6205e2434453db3c007a3c565b2ed, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:36.694714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:36.795631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1152/2294] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:36 2022] PUT /identity/v3/projects/e880544dafc140f089d4c475f37f4ec0/users/15a6205e2434453db3c007a3c565b2ed/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:36.807721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86706fd-25eb-4400-bfb0-7c74ce06023e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:36.810831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86706fd-25eb-4400-bfb0-7c74ce06023e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:36.810831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86706fd-25eb-4400-bfb0-7c74ce06023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:36.837172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c86706fd-25eb-4400-bfb0-7c74ce06023e None None] MFA Rules not processed for user `15a6205e2434453db3c007a3c565b2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:36.838847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c86706fd-25eb-4400-bfb0-7c74ce06023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:36.881101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:36.920030 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1143/2295] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:36.924978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:37.022410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad34e71-2fba-4d84-91f9-f6e7c7cfb651 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:37.029075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:37.030324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:37.030813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:37.031256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:37.032007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:37.034240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:37.133134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1153/2296] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:51:36 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:37.137247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86706fd-25eb-4400-bfb0-7c74ce06023e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:37.161784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86706fd-25eb-4400-bfb0-7c74ce06023e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:37.273786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:37.275123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:37.276590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:37.277089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:37.278103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:37.282089 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:37.407003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1144/2297] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:51:37 2022] POST /identity/v3/users => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:51:37.411115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:37.426890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24710b-8838-4212-8791-8297a05b424e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:37.434054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:37.434898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:37.435600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:37.436212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:37.441481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:37.443848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:37.462239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:37.463151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1154/2298] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:37.485884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c166a1-1db8-4478-8630-de9f675a9b07 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:37.491223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:37.492283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:37.492585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:37.492860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] PATH_INFO: `/v3/projects/8ed2174398764b0aa40fcbc4a4fa609b/users/9974e05c58f1439f94ed649605950443/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:37.526215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed2174398764b0aa40fcbc4a4fa609b, user_id=9974e05c58f1439f94ed649605950443, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:37.532151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:37.624603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1145/2299] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:37 2022] PUT /identity/v3/projects/8ed2174398764b0aa40fcbc4a4fa609b/users/9974e05c58f1439f94ed649605950443/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:51:37.629559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:37.666678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c6f34-5264-40a8-b630-cb2e467bf453 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:37.730057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:37.751158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e765d-5aee-4bdd-b1f0-526c47dbea6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:37.752268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e765d-5aee-4bdd-b1f0-526c47dbea6b admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:37.752729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e765d-5aee-4bdd-b1f0-526c47dbea6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:37.753159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e765d-5aee-4bdd-b1f0-526c47dbea6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:37.753903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e765d-5aee-4bdd-b1f0-526c47dbea6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:37.756056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e765d-5aee-4bdd-b1f0-526c47dbea6b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:37.800198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1155/2300] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:51:37.800349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:37.811829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53236d-1479-4c09-bf3a-506b3eea5977 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:37.828978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:37.830019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:37.830425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:37.832177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] PATH_INFO: `/v3/projects/8ed2174398764b0aa40fcbc4a4fa609b/users/9974e05c58f1439f94ed649605950443/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:37.835207 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed2174398764b0aa40fcbc4a4fa609b, user_id=9974e05c58f1439f94ed649605950443, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:37.838173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:37.856804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1146/2301] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:51:37 2022] PUT /identity/v3/projects/8ed2174398764b0aa40fcbc4a4fa609b/users/9974e05c58f1439f94ed649605950443/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:37.866089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fea64-6695-40b2-9f83-07e52a9a317b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:37.866390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fea64-6695-40b2-9f83-07e52a9a317b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:37.866678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fea64-6695-40b2-9f83-07e52a9a317b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:37.896837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f39fea64-6695-40b2-9f83-07e52a9a317b None None] MFA Rules not processed for user `9974e05c58f1439f94ed649605950443`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:37.898186 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f39fea64-6695-40b2-9f83-07e52a9a317b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:37.985054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1156/2302] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:37.994139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:37.996107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:38.091399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b59209-5d0c-408b-98e3-78e960fba803 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:38.096974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:38.098174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:38.098764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:38.099230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:38.189747 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:38.212179 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:38.213745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:38.226945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1147/2303] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:38.998906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d12d4-94ca-4b63-8426-d5f7cf2740d4 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:38.999388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d12d4-94ca-4b63-8426-d5f7cf2740d4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:38.999691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d12d4-94ca-4b63-8426-d5f7cf2740d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:39.066173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-850d12d4-94ca-4b63-8426-d5f7cf2740d4 None None] MFA Rules not processed for user `a638868e8f9949549380543e7538160d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:39.066774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-850d12d4-94ca-4b63-8426-d5f7cf2740d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:39.244791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1157/2304] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:39.272553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:39.275084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:39.391652 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711e8eb-309a-41c3-bdda-ea054ef7ba3b service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:39.398691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:39.399479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:39.399835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:39.400266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:39.402535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:39.473017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:39.474482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:39.484560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1148/2305] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:39.768004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:39.768305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:39.768627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:39.787056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:39.789019 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:39.841087 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:39.856253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1158/2306] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:51:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:39.871429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:39.873774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:39.969632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e77c7-08a9-458e-9923-91408bf12cf9 service glance] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:39.975308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7d33e-3915-41cf-b5a8-2832cd303824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:39.975968 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7d33e-3915-41cf-b5a8-2832cd303824 service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:39.976249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7d33e-3915-41cf-b5a8-2832cd303824 service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:39.976577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7d33e-3915-41cf-b5a8-2832cd303824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:39.985903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7d33e-3915-41cf-b5a8-2832cd303824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:39.987805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7d33e-3915-41cf-b5a8-2832cd303824 service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:39.997710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1149/2307] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:51:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:51:40.571937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:40.574453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:40.600493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee5a80-f548-4e86-bca8-c1182fb75450 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:40.616909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:40.618513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:40.620835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:40.621354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:40.785593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:40.787444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:40.830856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1159/2308] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:41.794499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d1b8a-3aa4-4b38-82d8-985c28d66db6 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:41.795657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d1b8a-3aa4-4b38-82d8-985c28d66db6 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:41.796093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d1b8a-3aa4-4b38-82d8-985c28d66db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:41.833222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-063d1b8a-3aa4-4b38-82d8-985c28d66db6 None None] MFA Rules not processed for user `3077a7934f9c4407a70a8790b8da9cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:41.836592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-063d1b8a-3aa4-4b38-82d8-985c28d66db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:41.896761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:41.961405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1150/2309] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:43.085278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:43.122207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f4ee-929e-4a7d-882b-66681aaefcd0 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:43.128859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:43.131124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:43.131124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:43.131124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:43.143266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:43.145583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:43.165762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1160/2310] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:46.862075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a5fd-29c4-450b-b1b2-8f05523033c7 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:46.862603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a5fd-29c4-450b-b1b2-8f05523033c7 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:46.863170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873a5fd-29c4-450b-b1b2-8f05523033c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:46.892270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-7873a5fd-29c4-450b-b1b2-8f05523033c7 None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:46.893667 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-7873a5fd-29c4-450b-b1b2-8f05523033c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:46.914234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1151/2311] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:46.929766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:46.962482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c165846-007c-4c34-a79a-09c7282701d4 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:46.974449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d990eb8-19e3-459f-822d-e3208a754b2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:46.975617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d990eb8-19e3-459f-822d-e3208a754b2d service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:46.976069 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d990eb8-19e3-459f-822d-e3208a754b2d service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:46.976620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d990eb8-19e3-459f-822d-e3208a754b2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:46.989643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d990eb8-19e3-459f-822d-e3208a754b2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:47.000287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d990eb8-19e3-459f-822d-e3208a754b2d service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:47.047101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1161/2312] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:51:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:51:54.004029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a5fd-29c4-450b-b1b2-8f05523033c7 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:54.173677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873a5fd-29c4-450b-b1b2-8f05523033c7 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:54.182941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:54.184155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:54.185140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:54.186370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] PATH_INFO: `/v3/users/3461ba437a0647b6afcdf1d68bba21b8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:54.187820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3461ba437a0647b6afcdf1d68bba21b8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:54.190440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:54.729563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3461ba437a0647b6afcdf1d68bba21b8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:54.772816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3461ba437a0647b6afcdf1d68bba21b8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:54.776905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3461ba437a0647b6afcdf1d68bba21b8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:54.780353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3461ba437a0647b6afcdf1d68bba21b8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:54.798954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3461ba437a0647b6afcdf1d68bba21b8'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:54.902053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1152/2313] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:54 2022] DELETE /identity/v3/users/3461ba437a0647b6afcdf1d68bba21b8 => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:54.902485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:54.903342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:54.903634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:54.953558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:54.955116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:55.142471 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1162/2314] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:55.155882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:55.159208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:55.276400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:55.294063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646c9c4-7498-4d71-a521-c1a5e83340b4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:55.298232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de808f83-54d6-455f-be3d-75462bfd76ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:55.298964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de808f83-54d6-455f-be3d-75462bfd76ef service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:55.299324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de808f83-54d6-455f-be3d-75462bfd76ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:55.299680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de808f83-54d6-455f-be3d-75462bfd76ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:55.316872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de808f83-54d6-455f-be3d-75462bfd76ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:55.318709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de808f83-54d6-455f-be3d-75462bfd76ef service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:55.339403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1153/2315] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:51:56.053690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:56.055169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:56.091084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230f22d-2e12-4403-b1fe-5f984c339472 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:56.111739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:56.112766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:56.113141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:56.113548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] PATH_INFO: `/v3/projects/4bbf837501614f9ab0fe56b4cd356663` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:56.114832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bbf837501614f9ab0fe56b4cd356663)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:56.118619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:56.272199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bbf837501614f9ab0fe56b4cd356663', 'request_id': 'req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:56.290010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bbf837501614f9ab0fe56b4cd356663', 'request_id': 'req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:56.290010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:56.295775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a07e-f341-48ff-81ef-7b803cba36de None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:56.296231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a07e-f341-48ff-81ef-7b803cba36de None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:56.296334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a07e-f341-48ff-81ef-7b803cba36de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:56.317399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c3b9a07e-f341-48ff-81ef-7b803cba36de None None] MFA Rules not processed for user `e72199904d9642e6bf9cacb7af568ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:51:56.319026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c3b9a07e-f341-48ff-81ef-7b803cba36de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:56.422126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bbf837501614f9ab0fe56b4cd356663', 'request_id': 'req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:56.439595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] The token cache is being invalidate because project 4bbf837501614f9ab0fe56b4cd356663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:56.439862 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:56.441324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:56.447460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1163/2316] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:56 2022] DELETE /identity/v3/projects/4bbf837501614f9ab0fe56b4cd356663 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:56.450453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:56.464589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:56.466020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1154/2317] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:51:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:51:56.640364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f76d3-88e4-4122-81e8-b2e4ffceb12e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:56.645781 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:56.646766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:56.647222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:56.647510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] PATH_INFO: `/v3/users/15a6205e2434453db3c007a3c565b2ed` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:56.649024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] RBAC: Authorizing `identity:delete_user(user_id=15a6205e2434453db3c007a3c565b2ed)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:56.651084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:57.435714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15a6205e2434453db3c007a3c565b2ed'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:57.441693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15a6205e2434453db3c007a3c565b2ed'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:57.441693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15a6205e2434453db3c007a3c565b2ed'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:57.457507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15a6205e2434453db3c007a3c565b2ed'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:57.571816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15a6205e2434453db3c007a3c565b2ed'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:57.616095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1164/2318] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:56 2022] DELETE /identity/v3/users/15a6205e2434453db3c007a3c565b2ed => generated 0 bytes in 1167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:58.364217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a07e-f341-48ff-81ef-7b803cba36de None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:58.395912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a07e-f341-48ff-81ef-7b803cba36de None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:58.473593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:58.474563 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:58.474922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:58.475277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] PATH_INFO: `/v3/projects/e880544dafc140f089d4c475f37f4ec0` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:58.476595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e880544dafc140f089d4c475f37f4ec0)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:58.478980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:58.629513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e880544dafc140f089d4c475f37f4ec0', 'request_id': 'req-338bb32e-b48e-44e8-91b8-956fd41d0e2a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:58.635202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e880544dafc140f089d4c475f37f4ec0', 'request_id': 'req-338bb32e-b48e-44e8-91b8-956fd41d0e2a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:58.643138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e880544dafc140f089d4c475f37f4ec0', 'request_id': 'req-338bb32e-b48e-44e8-91b8-956fd41d0e2a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:58.746382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] The token cache is being invalidate because project e880544dafc140f089d4c475f37f4ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:51:58.746807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:58.747685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:58.753107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1155/2319] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:51:58 2022] DELETE /identity/v3/projects/e880544dafc140f089d4c475f37f4ec0 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:51:58.754600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:51:58.762611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:51:58.786831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62e1f-aea2-4f1c-956d-cf90075c21be admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:51:58.889691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:51:58.889691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:51:58.889691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:51:58.889691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] PATH_INFO: `/v3/users/9974e05c58f1439f94ed649605950443` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:51:58.890082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9974e05c58f1439f94ed649605950443)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:51:58.892305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:51:58.908146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:51:59.205558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9974e05c58f1439f94ed649605950443'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:59.217253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9974e05c58f1439f94ed649605950443'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:59.221254 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9974e05c58f1439f94ed649605950443'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:59.240073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9974e05c58f1439f94ed649605950443'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:59.347959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-54927e90-aba3-43d0-88b2-9119d92b9e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9974e05c58f1439f94ed649605950443'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:51:59.380719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1165/2320] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:51:58 2022] DELETE /identity/v3/users/9974e05c58f1439f94ed649605950443 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:52:00.555304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:00.556848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:00.587275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb32e-b48e-44e8-91b8-956fd41d0e2a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:00.613748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:00.661944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:00.663375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:00.663669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:00.663950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] PATH_INFO: `/v3/projects/8ed2174398764b0aa40fcbc4a4fa609b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:00.665245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed2174398764b0aa40fcbc4a4fa609b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:00.667274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:00.976385 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed2174398764b0aa40fcbc4a4fa609b', 'request_id': 'req-63bd05e0-34b3-4696-8810-e65fbd74d2e7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:00.981278 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed2174398764b0aa40fcbc4a4fa609b', 'request_id': 'req-63bd05e0-34b3-4696-8810-e65fbd74d2e7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:01.026210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed2174398764b0aa40fcbc4a4fa609b', 'request_id': 'req-63bd05e0-34b3-4696-8810-e65fbd74d2e7'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:01.119719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] The token cache is being invalidate because project 8ed2174398764b0aa40fcbc4a4fa609b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:52:01.123813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:01.123813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-63bd05e0-34b3-4696-8810-e65fbd74d2e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:01.123813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1156/2321] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:52:00 2022] DELETE /identity/v3/projects/8ed2174398764b0aa40fcbc4a4fa609b => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:52:01.128426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f7215-23eb-4d70-be5d-45d4acad4ed4 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.129202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f7215-23eb-4d70-be5d-45d4acad4ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.129486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f7215-23eb-4d70-be5d-45d4acad4ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.152475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-806f7215-23eb-4d70-be5d-45d4acad4ed4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:01.153757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-806f7215-23eb-4d70-be5d-45d4acad4ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:01.228473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1166/2322] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:01.236478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3f68b-9962-4a79-9ff1-e49ddb7d0d92 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.236777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3f68b-9962-4a79-9ff1-e49ddb7d0d92 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.236972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3f68b-9962-4a79-9ff1-e49ddb7d0d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.266058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b7d3f68b-9962-4a79-9ff1-e49ddb7d0d92 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:01.267287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b7d3f68b-9962-4a79-9ff1-e49ddb7d0d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:01.281443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1157/2323] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:01.285944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f7215-23eb-4d70-be5d-45d4acad4ed4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:01.297541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f7215-23eb-4d70-be5d-45d4acad4ed4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:01.304410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:01.305361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.305894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.306352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.307305 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:52:01.307620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:01.309351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:01.329065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3f68b-9962-4a79-9ff1-e49ddb7d0d92 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:01.334964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1167/2324] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:52:01.339156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3f68b-9962-4a79-9ff1-e49ddb7d0d92 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:01.345703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:01.346558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.346914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.347438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.348230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:01.350341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:01.468506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1158/2325] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:52:01 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:52:01.473541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:01.496577 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce8abf-2dee-470b-9b76-5d23cb1d11c8 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:01.591645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:01.592783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.593681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.594183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.595159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:01.597531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:01.739944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1168/2326] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:52:01 2022] POST /identity/v3/users => generated 283 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:52:01.744245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:01.758028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ea5a1-7264-4126-b13b-40fb1637e9eb admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:01.762950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:01.764264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.764668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.765005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.765875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:01.767546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:01.778954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1159/2327] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:52:01.784023 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:01.790067 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:01.797562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d2fdc-2b4b-4e5a-8e9a-ed9982fdd201 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:01.807222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874585-7eb1-4fb6-8778-663a1606e4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:01.807222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874585-7eb1-4fb6-8778-663a1606e4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.807222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874585-7eb1-4fb6-8778-663a1606e4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.807222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874585-7eb1-4fb6-8778-663a1606e4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.807702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8874585-7eb1-4fb6-8778-663a1606e4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:01.809123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8874585-7eb1-4fb6-8778-663a1606e4c0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:01.825311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1169/2328] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:52:01.826552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:01.837239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faa700-c8e4-447e-af50-502d5fa44b4c admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:01.842325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:01.843269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:01.843700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:01.843981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] PATH_INFO: `/v3/projects/e0b4f1348a04474f8105fc69b6becd75/users/8c50ca2188e54edb90477cbb08d9f386/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:01.860107 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b4f1348a04474f8105fc69b6becd75, user_id=8c50ca2188e54edb90477cbb08d9f386, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:01.863005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:02.011293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1160/2329] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:52:01 2022] PUT /identity/v3/projects/e0b4f1348a04474f8105fc69b6becd75/users/8c50ca2188e54edb90477cbb08d9f386/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:52:02.021401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf06e-305d-407e-b81e-d4b86fa6d338 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:02.022138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf06e-305d-407e-b81e-d4b86fa6d338 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:02.022595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bf06e-305d-407e-b81e-d4b86fa6d338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:02.071134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-337bf06e-305d-407e-b81e-d4b86fa6d338 None None] MFA Rules not processed for user `8c50ca2188e54edb90477cbb08d9f386`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:02.071800 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-337bf06e-305d-407e-b81e-d4b86fa6d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:02.219758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1170/2330] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:52:02.231926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:02.234660 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:02.351763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:02.358139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a51d6b-bb62-450a-b26a-083d66fbe153 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:02.363255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:02.373249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:02.375934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:02.376228 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:02.394961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:02.396742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:02.407338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1161/2331] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:52:03.184896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:03.185215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:03.186518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:03.213822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:03.215238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:03.290287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1171/2332] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:52:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:03.307233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:03.309072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:03.431147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9589f6-76c6-4ce3-bb87-e5fc5e985c64 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:03.447836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f01fab-e6ca-4dc3-a2bc-3c1af5081a12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:03.448712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f01fab-e6ca-4dc3-a2bc-3c1af5081a12 service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:03.449065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f01fab-e6ca-4dc3-a2bc-3c1af5081a12 service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:03.449536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f01fab-e6ca-4dc3-a2bc-3c1af5081a12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:03.461000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f01fab-e6ca-4dc3-a2bc-3c1af5081a12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:03.462648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f01fab-e6ca-4dc3-a2bc-3c1af5081a12 service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:03.472821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1162/2333] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:52:06.218066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:06.221293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:06.342090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080f9f-fcdd-4daf-9b93-8cfd68e647dc None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:06.349637 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:06.351125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:06.355103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:06.355732 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] PATH_INFO: `/v3/users/1d0aa1ba95b240c0a1174f50a2c0d074` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:06.370588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0aa1ba95b240c0a1174f50a2c0d074)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:06.373968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:06.590724 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:06.715134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0aa1ba95b240c0a1174f50a2c0d074'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:06.715738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0aa1ba95b240c0a1174f50a2c0d074'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:06.716349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0aa1ba95b240c0a1174f50a2c0d074'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:06.726447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0aa1ba95b240c0a1174f50a2c0d074'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:06.816218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0aa1ba95b240c0a1174f50a2c0d074'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:06.835570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1172/2334] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:52:06 2022] DELETE /identity/v3/users/1d0aa1ba95b240c0a1174f50a2c0d074 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:52:06.840137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:06.841163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:06.842224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:06.862950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:06.864226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:06.946416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1163/2335] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:06.956608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:06.958500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:07.044728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b55355-e6df-4c4b-96d0-6e88944ae893 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:07.049263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520fb68-3334-4883-a780-74d25b5ecf00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:07.049907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520fb68-3334-4883-a780-74d25b5ecf00 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:07.050213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520fb68-3334-4883-a780-74d25b5ecf00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:07.050666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520fb68-3334-4883-a780-74d25b5ecf00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:07.061657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520fb68-3334-4883-a780-74d25b5ecf00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:07.063133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520fb68-3334-4883-a780-74d25b5ecf00 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:07.075179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1173/2336] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:52:07.599014 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:07.601262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:07.625113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea421b6f-122c-46db-91c7-e3c7807d6140 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:07.641439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:07.643441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:07.643963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:07.644492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] PATH_INFO: `/v3/projects/5ab9da65d4254e21912edde42a4a9b3b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:07.646234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab9da65d4254e21912edde42a4a9b3b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:07.648645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:07.823061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab9da65d4254e21912edde42a4a9b3b', 'request_id': 'req-5ceb9ecc-25cb-439d-80e0-a9354b57d610'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:07.838617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab9da65d4254e21912edde42a4a9b3b', 'request_id': 'req-5ceb9ecc-25cb-439d-80e0-a9354b57d610'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:07.867097 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab9da65d4254e21912edde42a4a9b3b', 'request_id': 'req-5ceb9ecc-25cb-439d-80e0-a9354b57d610'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:07.933288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] The token cache is being invalidate because project 5ab9da65d4254e21912edde42a4a9b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:52:07.933595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:07.934536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-5ceb9ecc-25cb-439d-80e0-a9354b57d610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:07.947526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1164/2337] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:52:07 2022] DELETE /identity/v3/projects/5ab9da65d4254e21912edde42a4a9b3b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:52:17.070894 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=45,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:17.935369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:24.115682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8027f-cb57-4d58-8e77-b44922c10374 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.119958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8027f-cb57-4d58-8e77-b44922c10374 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.119958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8027f-cb57-4d58-8e77-b44922c10374 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.119958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1174/2338] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:52:24.124133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee946a-eb15-4d18-b4ba-a731836293ac None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.124586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee946a-eb15-4d18-b4ba-a731836293ac None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.126197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee946a-eb15-4d18-b4ba-a731836293ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.145700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1fee946a-eb15-4d18-b4ba-a731836293ac None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:24.147055 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1fee946a-eb15-4d18-b4ba-a731836293ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:24.242779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1165/2339] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:52:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:24.254535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8027f-cb57-4d58-8e77-b44922c10374 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:24.256233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7fe8027f-cb57-4d58-8e77-b44922c10374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:24.344768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8027f-cb57-4d58-8e77-b44922c10374 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:24.348903 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c7302-1799-479c-9f34-f9d1547d9b82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:24.349563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c7302-1799-479c-9f34-f9d1547d9b82 service glance] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.349851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c7302-1799-479c-9f34-f9d1547d9b82 service glance] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.350130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c7302-1799-479c-9f34-f9d1547d9b82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.361007 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c7302-1799-479c-9f34-f9d1547d9b82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:24.362433 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c7302-1799-479c-9f34-f9d1547d9b82 service glance] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:24.373670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1175/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:52:24.543406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4436b66-dbe4-48d6-9567-10d5d4fd5380 None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.543766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4436b66-dbe4-48d6-9567-10d5d4fd5380 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.544061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4436b66-dbe4-48d6-9567-10d5d4fd5380 None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.546039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1166/2341] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:52:24.551155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a9ba0-1586-401e-91ce-42a1fb65dbdb None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.551396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a9ba0-1586-401e-91ce-42a1fb65dbdb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.551702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a9ba0-1586-401e-91ce-42a1fb65dbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.570715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4a0a9ba0-1586-401e-91ce-42a1fb65dbdb None None] MFA Rules not processed for user `d1d18a38f45e4501914e22689f145fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:24.571920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4a0a9ba0-1586-401e-91ce-42a1fb65dbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:24.638762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1176/2342] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:52:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:24.650471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4436b66-dbe4-48d6-9567-10d5d4fd5380 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:24.652145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c4436b66-dbe4-48d6-9567-10d5d4fd5380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:24.665226 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4436b66-dbe4-48d6-9567-10d5d4fd5380 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:24.669001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c30339-cca9-4a9f-aab8-56e1fe458f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:24.671218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c30339-cca9-4a9f-aab8-56e1fe458f2a service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.671218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c30339-cca9-4a9f-aab8-56e1fe458f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.671218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c30339-cca9-4a9f-aab8-56e1fe458f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.677900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c30339-cca9-4a9f-aab8-56e1fe458f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:24.679331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c30339-cca9-4a9f-aab8-56e1fe458f2a service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:24.687939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1167/2343] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:52:24.769243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd458963-8b33-41ff-9978-fe71c6eef3f3 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.769532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd458963-8b33-41ff-9978-fe71c6eef3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.769814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd458963-8b33-41ff-9978-fe71c6eef3f3 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.771225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1177/2344] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:52:24.776644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.776933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.777215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.796622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:24.797811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:24.806908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1168/2345] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:52:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:24.818234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd458963-8b33-41ff-9978-fe71c6eef3f3 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:24.828748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd458963-8b33-41ff-9978-fe71c6eef3f3 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:24.832227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0934e-4cd5-40d8-9bcb-e14b827faa93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:24.832884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0934e-4cd5-40d8-9bcb-e14b827faa93 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:24.833722 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0934e-4cd5-40d8-9bcb-e14b827faa93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:24.834008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0934e-4cd5-40d8-9bcb-e14b827faa93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:24.843995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0934e-4cd5-40d8-9bcb-e14b827faa93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:24.845438 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0934e-4cd5-40d8-9bcb-e14b827faa93 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:24.858843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1178/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:52:34.789954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=40 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:34.851807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=55 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:37.596594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:37.601997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:37.673047 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecdde9-efde-424e-a889-a25766fc11e0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:37.678224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:37.678873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:37.682481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:37.682481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:37.682481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:37.693058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:37.694686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:37.705077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1169/2347] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:52:45.303408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:45.303859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:45.303952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:45.322980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:45.324208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:45.379402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1179/2348] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:45.401219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:45.402636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:45.475737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407d811-6bf4-41d2-af1f-5907dfd602d3 service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:45.480736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4103c3f-102e-4f58-815e-52558e88b318 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:45.481384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4103c3f-102e-4f58-815e-52558e88b318 service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:45.481670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4103c3f-102e-4f58-815e-52558e88b318 service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:45.481950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4103c3f-102e-4f58-815e-52558e88b318 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:45.490202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4103c3f-102e-4f58-815e-52558e88b318 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:45.491791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4103c3f-102e-4f58-815e-52558e88b318 service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:45.500247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1170/2349] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:52:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:52:55.372945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=25 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:55.496837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=64 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:56.192138 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:56.193871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:56.265622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9139-5293-41cd-b934-6b447c66d258 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:56.270228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:56.271115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:56.271398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:56.271674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] PATH_INFO: `/v3/users/8c50ca2188e54edb90477cbb08d9f386` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:56.273058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c50ca2188e54edb90477cbb08d9f386)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:56.275021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:56.483214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c50ca2188e54edb90477cbb08d9f386'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:56.483948 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c50ca2188e54edb90477cbb08d9f386'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:56.484646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c50ca2188e54edb90477cbb08d9f386'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:56.492144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c50ca2188e54edb90477cbb08d9f386'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:56.558678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c50ca2188e54edb90477cbb08d9f386'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:56.583598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1180/2350] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:52:56 2022] DELETE /identity/v3/users/8c50ca2188e54edb90477cbb08d9f386 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:52:56.585323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:56.585855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:56.585938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:56.603935 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:56.605315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:56.669884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1171/2351] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:56.679834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:56.681342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:56.750656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc1f80-a762-414c-90ad-99dd26dd8916 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:56.754320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde2ff3-5377-4bc4-a63e-9af06cab97e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:56.754976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2ff3-5377-4bc4-a63e-9af06cab97e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:56.755266 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2ff3-5377-4bc4-a63e-9af06cab97e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:56.755546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2ff3-5377-4bc4-a63e-9af06cab97e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:56.765449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde2ff3-5377-4bc4-a63e-9af06cab97e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:56.766975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde2ff3-5377-4bc4-a63e-9af06cab97e0 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:56.777205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1181/2352] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:52:57.295391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:57.297120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:57.310738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba39d2-1d44-4639-832e-61bb8ee3d28b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:57.315535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:57.316758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:57.317249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:57.317496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] PATH_INFO: `/v3/projects/e0b4f1348a04474f8105fc69b6becd75` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:57.318881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b4f1348a04474f8105fc69b6becd75)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:57.320874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:57.428697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b4f1348a04474f8105fc69b6becd75', 'request_id': 'req-26f0b000-71f4-4f4a-aafb-717343d6e46a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:57.434704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b4f1348a04474f8105fc69b6becd75', 'request_id': 'req-26f0b000-71f4-4f4a-aafb-717343d6e46a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:57.442137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b4f1348a04474f8105fc69b6becd75', 'request_id': 'req-26f0b000-71f4-4f4a-aafb-717343d6e46a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:57.522070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] The token cache is being invalidate because project e0b4f1348a04474f8105fc69b6becd75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:52:57.522308 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:57.523046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-26f0b000-71f4-4f4a-aafb-717343d6e46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:52:57.524737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1172/2353] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:52:57 2022] DELETE /identity/v3/projects/e0b4f1348a04474f8105fc69b6becd75 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:52:57.530398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a096e-db76-49a7-bf21-ac08a3d6eeaf None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:57.537259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a096e-db76-49a7-bf21-ac08a3d6eeaf None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:57.537259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a096e-db76-49a7-bf21-ac08a3d6eeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:57.548875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-de1a096e-db76-49a7-bf21-ac08a3d6eeaf None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:57.550094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-de1a096e-db76-49a7-bf21-ac08a3d6eeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:57.578899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:57.637882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1182/2354] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:52:57.648247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704696a-e814-42f7-8b43-d3ef9942b43e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:57.649202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704696a-e814-42f7-8b43-d3ef9942b43e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:57.649493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704696a-e814-42f7-8b43-d3ef9942b43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:57.676551 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8704696a-e814-42f7-8b43-d3ef9942b43e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:57.678064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8704696a-e814-42f7-8b43-d3ef9942b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:57.687693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1173/2355] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:57.692084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a096e-db76-49a7-bf21-ac08a3d6eeaf None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:57.705091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a096e-db76-49a7-bf21-ac08a3d6eeaf None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:57.710037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:57.710714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:57.711005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:57.711290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:57.712032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:52:57.712348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:57.714159 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:57.727083 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1183/2356] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:52:57.732798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704696a-e814-42f7-8b43-d3ef9942b43e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:57.744081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704696a-e814-42f7-8b43-d3ef9942b43e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:57.749883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:57.750545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:57.750828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:57.751106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:57.751831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:57.753730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:57.854510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1174/2357] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:52:57 2022] POST /identity/v3/projects => generated 399 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:52:57.860194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:57.870728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88ea5f-5e1c-4845-b0c7-4816c41a96cf admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:57.946827 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:57.947587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:57.948003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:57.948484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:57.949203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:57.951916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:58.024891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1184/2358] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:52:57 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:52:58.026400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:58.049312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5df17-e0c7-40dd-8bfd-f11b93d031d6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:58.055857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:58.056929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:58.057470 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:58.063082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:58.063082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:58.063082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:58.077118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1175/2359] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:52:58.078728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:58.089548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfbcad-90e0-4f23-a6ec-d37ece50d1b6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:58.095518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979cc31-e824-4058-9e42-b37cb673a963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:58.113844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979cc31-e824-4058-9e42-b37cb673a963 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:58.113844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979cc31-e824-4058-9e42-b37cb673a963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:58.113844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979cc31-e824-4058-9e42-b37cb673a963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:58.113844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979cc31-e824-4058-9e42-b37cb673a963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:58.115236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979cc31-e824-4058-9e42-b37cb673a963 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:58.138877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:58.144272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1185/2360] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:52:58.152000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55decb7e-8583-4678-97b1-5f3837691e37 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:58.156971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:58.158471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:58.158910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:58.159544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] PATH_INFO: `/v3/projects/e668d4fed58445a086a673431c8f03dd/users/55ffebe8fb794d2a8b903061b9321809/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:58.225995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e668d4fed58445a086a673431c8f03dd, user_id=55ffebe8fb794d2a8b903061b9321809, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:58.228785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:58.587450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1176/2361] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:52:58 2022] PUT /identity/v3/projects/e668d4fed58445a086a673431c8f03dd/users/55ffebe8fb794d2a8b903061b9321809/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:52:58.656516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:58.657118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:58.657118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:58.684346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] MFA Rules not processed for user `55ffebe8fb794d2a8b903061b9321809`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:52:58.686464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:58.767053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1186/2362] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:52:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:52:58.774272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:52:58.776953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:52:58.804493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:52:58.850941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364ba1f-3007-4754-9bab-08228ffe2eb9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:52:58.853729 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:52:58.854411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:52:58.854728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:52:58.855042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:52:58.865433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:52:58.866904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:52:58.877583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1177/2363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:52:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:53:03.359473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:53:03.360995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:03.383420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:53:03.420038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774508e8-26a8-4e9e-ae36-bb92d733466e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:53:03.424162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8297cf5-47c7-45e3-b63c-b906c8507722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:53:03.424842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297cf5-47c7-45e3-b63c-b906c8507722 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:03.425133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297cf5-47c7-45e3-b63c-b906c8507722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:03.425418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297cf5-47c7-45e3-b63c-b906c8507722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:03.427183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c8297cf5-47c7-45e3-b63c-b906c8507722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:03.493655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8297cf5-47c7-45e3-b63c-b906c8507722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:53:03.495097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8297cf5-47c7-45e3-b63c-b906c8507722 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:53:03.505530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1187/2364] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:53:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:53:08.906851 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=17 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:53:09.113839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:53:09.187969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3326f-5293-4612-9cdc-93cf6f5b4c19 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:53:09.192561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:53:09.193440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:09.193720 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:09.193994 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] PATH_INFO: `/v3/users/55ffebe8fb794d2a8b903061b9321809` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:09.195980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ffebe8fb794d2a8b903061b9321809)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:53:09.197988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:53:09.453196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ffebe8fb794d2a8b903061b9321809'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:09.465236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ffebe8fb794d2a8b903061b9321809'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:09.465993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ffebe8fb794d2a8b903061b9321809'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:09.466675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ffebe8fb794d2a8b903061b9321809'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:09.473669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ffebe8fb794d2a8b903061b9321809'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:09.552253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1178/2365] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:53:09 2022] DELETE /identity/v3/users/55ffebe8fb794d2a8b903061b9321809 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:53:09.556711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:09.557588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:09.557868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:09.585462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:53:09.586571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:09.651636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1188/2366] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:53:09.662016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:53:09.663481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:09.729170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c427fc-565b-4293-a559-b07e2bc600c6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:53:09.733420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95002d-1d1a-4e90-bdc3-52538dc315d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:53:09.734069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95002d-1d1a-4e90-bdc3-52538dc315d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:09.734352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95002d-1d1a-4e90-bdc3-52538dc315d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:09.734634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95002d-1d1a-4e90-bdc3-52538dc315d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:09.743920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95002d-1d1a-4e90-bdc3-52538dc315d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:53:09.745416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95002d-1d1a-4e90-bdc3-52538dc315d3 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:53:09.755424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1179/2367] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:53:10.227987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:53:10.227987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:10.244919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67ea4d-eed2-48e0-a8fb-f729562cb141 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:53:10.251288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:53:10.252543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:10.252984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:10.253583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] PATH_INFO: `/v3/projects/e668d4fed58445a086a673431c8f03dd` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:10.256054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e668d4fed58445a086a673431c8f03dd)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:53:10.258971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:53:10.429861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e668d4fed58445a086a673431c8f03dd', 'request_id': 'req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:10.438247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e668d4fed58445a086a673431c8f03dd', 'request_id': 'req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:10.563979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e668d4fed58445a086a673431c8f03dd', 'request_id': 'req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:10.566457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:53:10.575518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] The token cache is being invalidate because project e668d4fed58445a086a673431c8f03dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:53:10.575929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:10.576612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-2e39fcdb-8e6f-4a5a-be60-a765ee3d6e8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:53:10.578204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1189/2368] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:53:10 2022] DELETE /identity/v3/projects/e668d4fed58445a086a673431c8f03dd => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:53:14.028966 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:14.032339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:14.032339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:14.046688 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] MFA Rules not processed for user `8daac4acc2fb4d47a7549ef710a70fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:53:14.047895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:14.067193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:53:14.108775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1180/2369] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:53:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:53:15.703123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008aaa47-3b38-4ba2-a684-c1daa251f84d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:15.703581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008aaa47-3b38-4ba2-a684-c1daa251f84d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:15.703675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008aaa47-3b38-4ba2-a684-c1daa251f84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:15.722573 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-008aaa47-3b38-4ba2-a684-c1daa251f84d None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:53:15.723898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-008aaa47-3b38-4ba2-a684-c1daa251f84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:15.784653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1190/2370] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jul 20 17:53:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:53:15.797271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:53:15.798694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:15.867499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4c3f8-374f-4f59-a6dc-56fc0f0f8c62 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:53:15.872603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dad91-db73-4610-b218-c4d3fd16042e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:53:15.873249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dad91-db73-4610-b218-c4d3fd16042e service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:15.873533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dad91-db73-4610-b218-c4d3fd16042e service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:15.873813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dad91-db73-4610-b218-c4d3fd16042e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:15.881893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dad91-db73-4610-b218-c4d3fd16042e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:53:15.883473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dad91-db73-4610-b218-c4d3fd16042e service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:53:15.891402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1181/2371] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:53:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:53:25.776787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=28,keystone:DELETE=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:53:25.888403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=47 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:53:26.707582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244836-ad69-4d0d-9776-ee76892ee661 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:26.708039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244836-ad69-4d0d-9776-ee76892ee661 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:26.708422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244836-ad69-4d0d-9776-ee76892ee661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:26.728137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4f244836-ad69-4d0d-9776-ee76892ee661 None None] MFA Rules not processed for user `2bc0d03b364946a5b8af3db4e9fd4780`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:53:26.729527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4f244836-ad69-4d0d-9776-ee76892ee661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:26.787265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1191/2372] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:53:36.778739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=25 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:53:42.514435 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:53:42.514948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:53:42.515050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:53:42.532984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] MFA Rules not processed for user `3077a7934f9c4407a70a8790b8da9cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:53:42.534114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:53:42.587659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1182/2373] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:53:52.581081 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=25 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:54:17.799733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd66ad-5d46-4118-aa23-4f455ffb276e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:17.800292 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd66ad-5d46-4118-aa23-4f455ffb276e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:17.800406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd66ad-5d46-4118-aa23-4f455ffb276e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:17.817687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b1cd66ad-5d46-4118-aa23-4f455ffb276e None None] MFA Rules not processed for user `311c19d7062648f78ba4c2090c58f14d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:54:17.818878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b1cd66ad-5d46-4118-aa23-4f455ffb276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:17.873322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1192/2374] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:54:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:54:24.165104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:24.167789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:24.247613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c97f-00bf-457e-b212-ac08dc2d3724 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:24.251361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:24.252611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:24.252896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:24.253176 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:24.255044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:24.349738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:24.351229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:24.378752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1183/2375] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:54:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:54:26.832750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd66ad-5d46-4118-aa23-4f455ffb276e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:26.844534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd66ad-5d46-4118-aa23-4f455ffb276e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:26.848964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:26.850066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:26.850357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:26.850639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:26.852555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:26.866598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:26.868034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:26.878957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1193/2376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:54:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:54:29.326191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:29.336713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57db04-4345-45cc-ae42-2b8e817e133a service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:29.340678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:29.341333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:29.341621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:29.341908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:29.345340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:29.359944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:29.361960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:29.371418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1184/2377] 10.222.0.42 () {60 vars in 1396 bytes} [Wed Jul 20 17:54:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:54:36.874057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=31 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:54:39.366917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=60 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:54:43.190174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:43.192322 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:43.260173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f9d7-223d-44e1-97f7-95277cd81a69 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:43.265024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:43.265893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:43.266184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:43.266464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] PATH_INFO: `/v3/users/2bc0d03b364946a5b8af3db4e9fd4780` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:43.267686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc0d03b364946a5b8af3db4e9fd4780)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:43.269698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:43.456927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc0d03b364946a5b8af3db4e9fd4780'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:43.457593 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc0d03b364946a5b8af3db4e9fd4780'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:43.458201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc0d03b364946a5b8af3db4e9fd4780'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:43.466057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc0d03b364946a5b8af3db4e9fd4780'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:43.529900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc0d03b364946a5b8af3db4e9fd4780'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:43.548661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1194/2378] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:54:43 2022] DELETE /identity/v3/users/2bc0d03b364946a5b8af3db4e9fd4780 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:54:43.845047 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:43.855813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104701-eab9-4f14-931a-0334c3733081 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:43.915466 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:43.916359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:43.916675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:43.916950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] PATH_INFO: `/v3/projects/10e93d9f177d4ed5bba338ba68547bf7` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:43.918131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e93d9f177d4ed5bba338ba68547bf7)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:43.920045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:44.037170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e93d9f177d4ed5bba338ba68547bf7', 'request_id': 'req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:44.042537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e93d9f177d4ed5bba338ba68547bf7', 'request_id': 'req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:44.048605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e93d9f177d4ed5bba338ba68547bf7', 'request_id': 'req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:44.135137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] The token cache is being invalidate because project 10e93d9f177d4ed5bba338ba68547bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:54:44.135429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:44.136057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:54:44.137632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1185/2379] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:54:43 2022] DELETE /identity/v3/projects/10e93d9f177d4ed5bba338ba68547bf7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:54:47.392293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:47.393828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:47.461798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6f8df-d987-4aed-9fb2-a93b56010a38 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:47.495329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:54:47.511736 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:47.512592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:47.512883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:47.513197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:47.515020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:47.524614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:47.526090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:47.535441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1195/2380] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:54:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:54:49.435801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:49.437195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:49.517615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4daa9-a0a2-4dee-88c0-f78f42d5d1db admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:49.523820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1f82f-7b52-4246-8666-320ef2b58c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:49.524522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1f82f-7b52-4246-8666-320ef2b58c8d service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:49.524812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1f82f-7b52-4246-8666-320ef2b58c8d service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:49.525093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1f82f-7b52-4246-8666-320ef2b58c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:49.526885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-abe1f82f-7b52-4246-8666-320ef2b58c8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:49.556500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:54:49.589788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1f82f-7b52-4246-8666-320ef2b58c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:49.591433 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1f82f-7b52-4246-8666-320ef2b58c8d service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:49.601048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1186/2381] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:54:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:54:53.716745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:53.718162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:53.734483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71356fb8-0dae-461a-b86a-fc95fb511336 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:53.739871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f69c6-92ee-4068-bc27-c6dc24880a50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:53.740563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f69c6-92ee-4068-bc27-c6dc24880a50 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:53.740850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f69c6-92ee-4068-bc27-c6dc24880a50 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:53.741127 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f69c6-92ee-4068-bc27-c6dc24880a50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:53.742803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a51f69c6-92ee-4068-bc27-c6dc24880a50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:53.754489 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f69c6-92ee-4068-bc27-c6dc24880a50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:53.756058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f69c6-92ee-4068-bc27-c6dc24880a50 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:53.766459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1196/2382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:54:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:54:55.925903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:55.926910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:55.927023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:55.944754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] MFA Rules not processed for user `8daac4acc2fb4d47a7549ef710a70fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:54:55.945971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:55.999295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1187/2383] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:54:56.826486 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8c116-40d7-48e9-bdd0-a9920852924d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:56.826942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8c116-40d7-48e9-bdd0-a9920852924d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:56.827032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8c116-40d7-48e9-bdd0-a9920852924d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:56.845465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-32e8c116-40d7-48e9-bdd0-a9920852924d None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:54:56.846661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-32e8c116-40d7-48e9-bdd0-a9920852924d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:56.903245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1197/2384] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jul 20 17:54:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:54:57.060537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:54:57.062589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:54:57.134900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a74b7-9c39-4f8b-bcbc-a047fd7469f5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:54:57.140166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:54:57.140850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:54:57.141140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:54:57.141421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:54:57.149464 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:54:57.151101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:54:57.159352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1188/2385] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:54:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:55:01.820900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8c116-40d7-48e9-bdd0-a9920852924d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:01.837850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8c116-40d7-48e9-bdd0-a9920852924d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:01.846053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:01.852085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:01.854231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:01.854673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:01.860132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:01.898420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:01.902276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:01.919332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1198/2386] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:03.161572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:03.164604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:03.250238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:03.254716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea030184-a039-4a11-864d-7bb4fa38e7d7 service swift] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:03.262082 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:03.262854 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:03.263206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:03.274009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:03.278849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:03.292213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:03.293872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:03.312209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1189/2387] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:55:06.867873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:06.869799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:06.934196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c9329-d72c-4558-8f34-e9fe9c5905be service nova] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:06.938675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:06.939540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:06.940090 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:06.940364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] PATH_INFO: `/v3/users/e72199904d9642e6bf9cacb7af568ddb` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:06.941580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72199904d9642e6bf9cacb7af568ddb)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:06.943524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:07.120065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72199904d9642e6bf9cacb7af568ddb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.126205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72199904d9642e6bf9cacb7af568ddb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.126205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72199904d9642e6bf9cacb7af568ddb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.129897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72199904d9642e6bf9cacb7af568ddb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.132725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:07.231824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72199904d9642e6bf9cacb7af568ddb'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.249374 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1199/2388] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:55:06 2022] DELETE /identity/v3/users/e72199904d9642e6bf9cacb7af568ddb => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:55:07.520705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:07.531874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7825a-120e-47bf-8d51-b174e51620ad service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:07.588941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:07.589795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:07.590072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:07.590341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] PATH_INFO: `/v3/projects/63d0c5b02155444fb44a732c46674b8f` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:07.591529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d0c5b02155444fb44a732c46674b8f)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:07.593459 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:07.707299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d0c5b02155444fb44a732c46674b8f', 'request_id': 'req-8e819894-58a9-4635-8d60-587f705d94c9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.713724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d0c5b02155444fb44a732c46674b8f', 'request_id': 'req-8e819894-58a9-4635-8d60-587f705d94c9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.719848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d0c5b02155444fb44a732c46674b8f', 'request_id': 'req-8e819894-58a9-4635-8d60-587f705d94c9'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.830158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] The token cache is being invalidate because project 63d0c5b02155444fb44a732c46674b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:55:07.830378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.831009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:07.832603 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1190/2389] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:55:07 2022] DELETE /identity/v3/projects/63d0c5b02155444fb44a732c46674b8f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:55:07.837262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:07.838740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:07.851629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdd407-dd94-4b9a-a477-39a39b7d41e6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:07.908851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:07.909766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:07.910055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:07.910339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] PATH_INFO: `/v3/users/a638868e8f9949549380543e7538160d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:07.911585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a638868e8f9949549380543e7538160d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:07.913591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:08.138029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a638868e8f9949549380543e7538160d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.138691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a638868e8f9949549380543e7538160d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.139302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a638868e8f9949549380543e7538160d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.146256 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a638868e8f9949549380543e7538160d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.249431 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc0275f5-d344-4b7c-9b61-e2256c472751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a638868e8f9949549380543e7538160d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.269396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1200/2390] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:55:07 2022] DELETE /identity/v3/users/a638868e8f9949549380543e7538160d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:55:08.512731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:08.518439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:08.529116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819894-58a9-4635-8d60-587f705d94c9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:08.590225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:08.591093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:08.591378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:08.591656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] PATH_INFO: `/v3/projects/5acd24ef057e49868df3d27bfeb3f51d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:08.592873 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5acd24ef057e49868df3d27bfeb3f51d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:08.594809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:08.620475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:08.725309 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5acd24ef057e49868df3d27bfeb3f51d', 'request_id': 'req-28ceeade-b092-4f7f-8679-c34601013fe6'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.731367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5acd24ef057e49868df3d27bfeb3f51d', 'request_id': 'req-28ceeade-b092-4f7f-8679-c34601013fe6'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.737534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5acd24ef057e49868df3d27bfeb3f51d', 'request_id': 'req-28ceeade-b092-4f7f-8679-c34601013fe6'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.838152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] The token cache is being invalidate because project 5acd24ef057e49868df3d27bfeb3f51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:55:08.838504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.839136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-28ceeade-b092-4f7f-8679-c34601013fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:55:08.840757 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1191/2391] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:55:08 2022] DELETE /identity/v3/projects/5acd24ef057e49868df3d27bfeb3f51d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:55:08.845900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56284-9fd8-4cca-8814-100f1ccb4a7b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:08.846275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56284-9fd8-4cca-8814-100f1ccb4a7b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:08.846635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56284-9fd8-4cca-8814-100f1ccb4a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:08.868222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-34d56284-9fd8-4cca-8814-100f1ccb4a7b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:08.869494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-34d56284-9fd8-4cca-8814-100f1ccb4a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:08.919470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:08.931618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1201/2392] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:55:08.939471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cbd62-048b-45ac-a302-5b73eb0f6a03 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:08.940023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cbd62-048b-45ac-a302-5b73eb0f6a03 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:08.940314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cbd62-048b-45ac-a302-5b73eb0f6a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:08.958395 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-333cbd62-048b-45ac-a302-5b73eb0f6a03 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:08.960139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-333cbd62-048b-45ac-a302-5b73eb0f6a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:08.968178 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1192/2393] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:08.972273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56284-9fd8-4cca-8814-100f1ccb4a7b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:08.982613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56284-9fd8-4cca-8814-100f1ccb4a7b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:08.987408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:08.988071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:08.988360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:08.988679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:08.989414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:55:08.989725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:08.991354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:09.003117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1202/2394] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:55:09.007632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cbd62-048b-45ac-a302-5b73eb0f6a03 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:09.018026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cbd62-048b-45ac-a302-5b73eb0f6a03 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:09.022957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:09.026166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:09.026166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:09.026166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:09.026166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:09.026891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:09.141632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1193/2395] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:55:09 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:55:09.142847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:09.152788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cba71-bea0-46e2-8fd4-a842d77de9a3 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:09.213165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:09.213886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:09.214182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:09.214464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:09.215223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:09.217301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:09.302013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1203/2396] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:55:09 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:55:09.306701 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:09.318204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec8d81-a21c-48e5-9c72-28d28117777f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:09.322897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:09.323579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:09.323868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:09.324147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:09.324944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:09.326576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:09.344297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1194/2397] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:55:09.348813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:09.359661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617c3fc-58db-471f-84d7-e95ad8ad2bbd admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:09.365754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ebbde-86e0-47b4-8bc9-d10edee70276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:09.366424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ebbde-86e0-47b4-8bc9-d10edee70276 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:09.366757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ebbde-86e0-47b4-8bc9-d10edee70276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:09.367037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ebbde-86e0-47b4-8bc9-d10edee70276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:09.367848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ebbde-86e0-47b4-8bc9-d10edee70276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:09.369741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ebbde-86e0-47b4-8bc9-d10edee70276 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:09.388062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1204/2398] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:55:09.392300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:09.404007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d3d49-0890-47e8-a77a-838fb0374524 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:09.409533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:09.410532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:09.410820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:09.411101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] PATH_INFO: `/v3/projects/863e2e49ffb74a889fb88c4c2e7edcd4/users/0fcc8275481e41f38bbdfac7db67b2e5/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:09.430925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=863e2e49ffb74a889fb88c4c2e7edcd4, user_id=0fcc8275481e41f38bbdfac7db67b2e5, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:09.442058 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:09.547681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1195/2399] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:55:09 2022] PUT /identity/v3/projects/863e2e49ffb74a889fb88c4c2e7edcd4/users/0fcc8275481e41f38bbdfac7db67b2e5/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:55:09.553113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e774d00-0bb9-44c0-987b-37b59908b9b6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:09.553432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e774d00-0bb9-44c0-987b-37b59908b9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:09.553765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e774d00-0bb9-44c0-987b-37b59908b9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:09.572967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-0e774d00-0bb9-44c0-987b-37b59908b9b6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:09.574494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0e774d00-0bb9-44c0-987b-37b59908b9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:09.638487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1205/2400] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:55:09.648776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:09.650473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:09.711005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1b91-0dc5-4280-9f1c-d11f775cf950 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:09.714729 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cba29c-5630-4a54-9379-9d4bc93df617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:09.715389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cba29c-5630-4a54-9379-9d4bc93df617 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:09.715677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cba29c-5630-4a54-9379-9d4bc93df617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:09.715951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cba29c-5630-4a54-9379-9d4bc93df617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:09.726740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cba29c-5630-4a54-9379-9d4bc93df617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:09.728240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cba29c-5630-4a54-9379-9d4bc93df617 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:09.738361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1196/2401] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:15.006655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad517-3c73-4a66-b1b2-b71cf62b02ad None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:15.007124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad517-3c73-4a66-b1b2-b71cf62b02ad None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:15.007219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ad517-3c73-4a66-b1b2-b71cf62b02ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:15.025604 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-db8ad517-3c73-4a66-b1b2-b71cf62b02ad None None] MFA Rules not processed for user `3077a7934f9c4407a70a8790b8da9cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:15.027112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-db8ad517-3c73-4a66-b1b2-b71cf62b02ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:15.132251 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1206/2402] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:15.428410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c1da-dd24-4b54-9e66-b6ae3ea9e817 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:15.428930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c1da-dd24-4b54-9e66-b6ae3ea9e817 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:15.429051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c1da-dd24-4b54-9e66-b6ae3ea9e817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:15.478118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f358c1da-dd24-4b54-9e66-b6ae3ea9e817 None None] MFA Rules not processed for user `0fcc8275481e41f38bbdfac7db67b2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:15.479350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f358c1da-dd24-4b54-9e66-b6ae3ea9e817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:15.549559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:15.578731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ad517-3c73-4a66-b1b2-b71cf62b02ad None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:15.592480 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1197/2403] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:15.592674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ad517-3c73-4a66-b1b2-b71cf62b02ad None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:15.594405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358c1da-dd24-4b54-9e66-b6ae3ea9e817 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:15.595913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:15.596875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:15.597245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:15.597601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:15.599779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:15.605284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358c1da-dd24-4b54-9e66-b6ae3ea9e817 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:15.608614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:15.611035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:15.611406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:15.611709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:15.612085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:15.612294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:15.613032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:15.618819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:15.619061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:15.625225 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1207/2404] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:55:15.724469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1198/2405] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:55:15 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:55:15.726380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:15.739215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebef9fc-8594-48d5-9f98-65c4dea5fcc2 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:15.806366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:15.807061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:15.807356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:15.807650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:15.808676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:15.810733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:15.906166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1208/2406] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:55:15 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:55:15.910949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:15.918830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988952a-d28a-4a89-b45d-b1f752d860e7 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:15.924460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:15.925145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:15.925432 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:15.925710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:15.926452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:15.928148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:15.941354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1199/2407] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:55:15.946160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:15.969032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92898a1-edf2-49bc-ad33-635b924d578c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:15.975081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:15.976136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:15.981085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:15.981085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] PATH_INFO: `/v3/projects/77aec6c344774102b7f54cff8e9667e0/users/a59db19c004e4f35b930fc4739cd2274/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:16.001199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77aec6c344774102b7f54cff8e9667e0, user_id=a59db19c004e4f35b930fc4739cd2274, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:16.003854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:16.071792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:16.073821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:16.120457 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1209/2408] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:55:15 2022] PUT /identity/v3/projects/77aec6c344774102b7f54cff8e9667e0/users/a59db19c004e4f35b930fc4739cd2274/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:55:16.121410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:16.171645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3625fc-4bec-4ccd-9e82-305191d4fbbd admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:16.209428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf61b7-0d6a-4114-bb13-222960554373 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:16.219275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:16.219275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:16.219275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:16.219275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:16.219740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:16.219740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:16.230744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1210/2409] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:55:16.234804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:16.241953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:16.242982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:16.244138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:16.244138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:16.246970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:16.247582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3969e8-5568-4779-9f09-1616b8b8afe9 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:16.252448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ef033-b1c2-49e7-a87d-3b2a95cea560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:16.253356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef033-b1c2-49e7-a87d-3b2a95cea560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:16.253643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef033-b1c2-49e7-a87d-3b2a95cea560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:16.253928 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef033-b1c2-49e7-a87d-3b2a95cea560 admin admin] PATH_INFO: `/v3/projects/77aec6c344774102b7f54cff8e9667e0/users/a59db19c004e4f35b930fc4739cd2274/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:16.256545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ef033-b1c2-49e7-a87d-3b2a95cea560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77aec6c344774102b7f54cff8e9667e0, user_id=a59db19c004e4f35b930fc4739cd2274, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:16.259215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ef033-b1c2-49e7-a87d-3b2a95cea560 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:16.260636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:16.262381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:16.270997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1211/2410] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:55:16 2022] PUT /identity/v3/projects/77aec6c344774102b7f54cff8e9667e0/users/a59db19c004e4f35b930fc4739cd2274/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:55:16.277377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1200/2411] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:23.804363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb77d48-f677-4e0b-8231-18dbf6fd9f16 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:23.804899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb77d48-f677-4e0b-8231-18dbf6fd9f16 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:23.805046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb77d48-f677-4e0b-8231-18dbf6fd9f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:23.834829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-bcb77d48-f677-4e0b-8231-18dbf6fd9f16 None None] MFA Rules not processed for user `a59db19c004e4f35b930fc4739cd2274`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:23.836271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bcb77d48-f677-4e0b-8231-18dbf6fd9f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:23.872065 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:23.912005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1212/2412] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:55:23 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:23.925100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:23.926532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:23.969766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:23.998985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754c022-3065-4900-93e5-4a04b1e3db49 service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:24.062929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:24.063584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:24.063867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:24.064143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:24.120307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:24.121939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:24.130750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1201/2413] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:26.971460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c1be-9362-4fab-ba4b-c84267239b48 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:26.971950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c1be-9362-4fab-ba4b-c84267239b48 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:26.972063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c1be-9362-4fab-ba4b-c84267239b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:26.991808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3791c1be-9362-4fab-ba4b-c84267239b48 None None] MFA Rules not processed for user `0fcc8275481e41f38bbdfac7db67b2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:26.993658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3791c1be-9362-4fab-ba4b-c84267239b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:27.005276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1213/2414] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:55:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:55:27.634134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:27.636406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:27.652695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38174c-b303-496a-a10d-c62f79d2cd4c service nova] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:27.658709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:27.659318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:27.659602 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:27.659905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:27.661774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:27.675808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:27.677721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:27.689175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1202/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:31.820034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425dc2-32fa-4d65-8ecb-210e07f66e31 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:31.820541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425dc2-32fa-4d65-8ecb-210e07f66e31 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:31.820632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425dc2-32fa-4d65-8ecb-210e07f66e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:31.839575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-7a425dc2-32fa-4d65-8ecb-210e07f66e31 None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:31.841199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-7a425dc2-32fa-4d65-8ecb-210e07f66e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:31.901126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1214/2416] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:55:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:31.913918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:31.915409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:31.993085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:31.997765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bbda9-9530-4f5b-92db-e1bd782f1002 service nova] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:32.003009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb38a2-54f4-40bb-9017-c11dfded6588 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:32.003682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb38a2-54f4-40bb-9017-c11dfded6588 service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:32.003952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb38a2-54f4-40bb-9017-c11dfded6588 service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:32.005331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb38a2-54f4-40bb-9017-c11dfded6588 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:32.019407 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb38a2-54f4-40bb-9017-c11dfded6588 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:32.021674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb38a2-54f4-40bb-9017-c11dfded6588 service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:32.032022 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1203/2417] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:55:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:55:41.893826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=43 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:42.029284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:55:53.130282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1c62-4ecc-4340-a131-23a078f9ea05 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.130814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1c62-4ecc-4340-a131-23a078f9ea05 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.130959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1c62-4ecc-4340-a131-23a078f9ea05 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.133475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1215/2418] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:55:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:55:53.139682 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b185e-5d85-4b39-8563-d231c0742cd8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.140196 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b185e-5d85-4b39-8563-d231c0742cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.141011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b185e-5d85-4b39-8563-d231c0742cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.159173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8d7b185e-5d85-4b39-8563-d231c0742cd8 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:53.160588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8d7b185e-5d85-4b39-8563-d231c0742cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:53.170998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1204/2419] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:53.186767 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1c62-4ecc-4340-a131-23a078f9ea05 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:53.188572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5eae1c62-4ecc-4340-a131-23a078f9ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:53.248505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1c62-4ecc-4340-a131-23a078f9ea05 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:53.252782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7386f-58e4-4070-ab8d-9ca687da0134 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1a294634524a9d8ff48c7c50726228', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:53.258028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7386f-58e4-4070-ab8d-9ca687da0134 service glance] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.258028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7386f-58e4-4070-ab8d-9ca687da0134 service glance] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.258028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7386f-58e4-4070-ab8d-9ca687da0134 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.265116 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7386f-58e4-4070-ab8d-9ca687da0134 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:53.266581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7386f-58e4-4070-ab8d-9ca687da0134 service glance] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:53.278293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1216/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:53.528833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d39fb-8f04-4fd2-9b23-621a644818fb None None] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.529123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d39fb-8f04-4fd2-9b23-621a644818fb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.529403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d39fb-8f04-4fd2-9b23-621a644818fb None None] PATH_INFO: `/` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.530776 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1205/2421] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:55:53.535504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89d2c-05af-4012-875a-f4ee97e5f6d5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.535826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89d2c-05af-4012-875a-f4ee97e5f6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.536114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b89d2c-05af-4012-875a-f4ee97e5f6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.555357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c1b89d2c-05af-4012-875a-f4ee97e5f6d5 None None] MFA Rules not processed for user `d1d18a38f45e4501914e22689f145fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:53.556628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c1b89d2c-05af-4012-875a-f4ee97e5f6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:53.606883 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1217/2422] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:53.618866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d39fb-8f04-4fd2-9b23-621a644818fb None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:53.620611 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-737d39fb-8f04-4fd2-9b23-621a644818fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:53.633381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d39fb-8f04-4fd2-9b23-621a644818fb None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:53.637513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f78cc7-e200-4897-b451-bbf239672fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:53.638143 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f78cc7-e200-4897-b451-bbf239672fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.638424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f78cc7-e200-4897-b451-bbf239672fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.638697 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f78cc7-e200-4897-b451-bbf239672fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.645638 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f78cc7-e200-4897-b451-bbf239672fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:53.647553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f78cc7-e200-4897-b451-bbf239672fd2 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:53.655095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1206/2423] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:53.743295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8f01a-7f25-424f-9897-b83af886ffc7 None None] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.743590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8f01a-7f25-424f-9897-b83af886ffc7 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.743872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8f01a-7f25-424f-9897-b83af886ffc7 None None] PATH_INFO: `/` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.745334 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1218/2424] 10.222.0.42 () {54 vars in 921 bytes} [Wed Jul 20 17:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 17:55:53.751210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7e1d-800d-4e0c-a695-01c8bc85e4f5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.752831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7e1d-800d-4e0c-a695-01c8bc85e4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.752831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7e1d-800d-4e0c-a695-01c8bc85e4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.769695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-93ce7e1d-800d-4e0c-a695-01c8bc85e4f5 None None] MFA Rules not processed for user `b6f536917f434773a052aa074d5aa07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:53.770886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-93ce7e1d-800d-4e0c-a695-01c8bc85e4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:53.780691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1207/2425] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Jul 20 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:53.791837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8f01a-7f25-424f-9897-b83af886ffc7 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:53.803362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8f01a-7f25-424f-9897-b83af886ffc7 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:53.806956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:53.807587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:53.807875 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:53.808197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:53.818803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:53.820288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:53.831073 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1219/2426] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:55:59.233434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607a2fd-4134-471f-b975-9f0aeb299617 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:59.233979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607a2fd-4134-471f-b975-9f0aeb299617 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:59.234175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607a2fd-4134-471f-b975-9f0aeb299617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:59.252584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b607a2fd-4134-471f-b975-9f0aeb299617 None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:55:59.253814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b607a2fd-4134-471f-b975-9f0aeb299617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:55:59.261057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1208/2427] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 17:55:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:55:59.282806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:55:59.302196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e515c1-94f4-4b52-a89c-ff35ff38f625 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:55:59.302196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:55:59.302196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:55:59.302196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:55:59.302711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:55:59.310564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:55:59.312182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:55:59.321117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1220/2428] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:55:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:56:02.009842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607a2fd-4134-471f-b975-9f0aeb299617 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:02.023206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607a2fd-4134-471f-b975-9f0aeb299617 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:02.028694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34da29-2542-4d6d-8760-797e01f5099e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:02.029347 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34da29-2542-4d6d-8760-797e01f5099e service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:02.029631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34da29-2542-4d6d-8760-797e01f5099e service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:02.029907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34da29-2542-4d6d-8760-797e01f5099e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:02.031676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4e34da29-2542-4d6d-8760-797e01f5099e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:02.092743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34da29-2542-4d6d-8760-797e01f5099e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:02.094959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34da29-2542-4d6d-8760-797e01f5099e service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:02.106784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1209/2429] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:09.317558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=61 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:11.577134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:11.591769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1962b-4492-4d7b-9aac-f7b7ba2e05ec service swift] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:11.602757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:11.603561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:11.603899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:11.604350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:11.616130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:11.618130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:11.628986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1221/2430] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:12.124414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=43 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:15.674485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621187-7233-4ef2-bee1-217b14e70b33 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:15.675062 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621187-7233-4ef2-bee1-217b14e70b33 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:15.677362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61621187-7233-4ef2-bee1-217b14e70b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:15.695155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-61621187-7233-4ef2-bee1-217b14e70b33 None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:15.696605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-61621187-7233-4ef2-bee1-217b14e70b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:15.704698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1210/2431] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jul 20 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:15.718494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:15.731027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710078e-8de4-4124-ba97-60ac88a679f4 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:15.736121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:15.736951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:15.737429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:15.741206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:15.747307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:15.748921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:15.756960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1222/2432] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 17:56:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 17:56:17.444257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621187-7233-4ef2-bee1-217b14e70b33 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:17.456029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61621187-7233-4ef2-bee1-217b14e70b33 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:17.460486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:17.463972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:17.463972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:17.463972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:17.463972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:17.529626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:17.531071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:17.541088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1211/2433] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:20.934876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:20.939188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:20.951402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10087e32-aea9-4def-a2ea-86ae1e705372 service swift] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:20.955892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:20.956831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:20.957117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:20.957389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] PATH_INFO: `/v3/users/8daac4acc2fb4d47a7549ef710a70fca` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:20.958567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8daac4acc2fb4d47a7549ef710a70fca)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:20.961020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:21.149763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8daac4acc2fb4d47a7549ef710a70fca'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.151905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8daac4acc2fb4d47a7549ef710a70fca'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.152749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8daac4acc2fb4d47a7549ef710a70fca'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.161574 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8daac4acc2fb4d47a7549ef710a70fca'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.243213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fed9317f-ad89-4d00-a3cc-b5656bc6c8b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8daac4acc2fb4d47a7549ef710a70fca'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.261447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1223/2434] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:20 2022] DELETE /identity/v3/users/8daac4acc2fb4d47a7549ef710a70fca => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:21.494431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:21.504313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e5475-3c47-40f6-8582-c1ca6188f97c service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:21.561267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:21.562133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:21.562451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:21.562734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] PATH_INFO: `/v3/projects/bde6b55e64864d188ee5fafc4b9edc4a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:21.563949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde6b55e64864d188ee5fafc4b9edc4a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:21.565982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:21.666748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde6b55e64864d188ee5fafc4b9edc4a', 'request_id': 'req-3a74eda5-f9e7-47e1-a338-46bbecb39466'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.672319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde6b55e64864d188ee5fafc4b9edc4a', 'request_id': 'req-3a74eda5-f9e7-47e1-a338-46bbecb39466'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.679085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde6b55e64864d188ee5fafc4b9edc4a', 'request_id': 'req-3a74eda5-f9e7-47e1-a338-46bbecb39466'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.740288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] The token cache is being invalidate because project bde6b55e64864d188ee5fafc4b9edc4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:21.741193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.741821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3a74eda5-f9e7-47e1-a338-46bbecb39466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:21.743362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1212/2435] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:21 2022] DELETE /identity/v3/projects/bde6b55e64864d188ee5fafc4b9edc4a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:21.748418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b65803-3e80-40e9-9389-1354b10fe9c5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:21.748729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b65803-3e80-40e9-9389-1354b10fe9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:21.749009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b65803-3e80-40e9-9389-1354b10fe9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:21.766263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-68b65803-3e80-40e9-9389-1354b10fe9c5 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:21.767440 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-68b65803-3e80-40e9-9389-1354b10fe9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:21.826330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1224/2436] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:21.834160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f172920-74b8-425d-9d01-45720d34cac2 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:21.834481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f172920-74b8-425d-9d01-45720d34cac2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:21.834788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f172920-74b8-425d-9d01-45720d34cac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:21.853362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8f172920-74b8-425d-9d01-45720d34cac2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:21.854582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8f172920-74b8-425d-9d01-45720d34cac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:21.863194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1213/2437] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:21.867991 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b65803-3e80-40e9-9389-1354b10fe9c5 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:21.877697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b65803-3e80-40e9-9389-1354b10fe9c5 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:21.882258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:21.882909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:21.883195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:21.883496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:21.884223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:56:21.884562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:21.886176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:21.898242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1225/2438] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:56:21.902545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f172920-74b8-425d-9d01-45720d34cac2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:21.911833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f172920-74b8-425d-9d01-45720d34cac2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:21.917250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:21.917908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:21.918192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:21.918469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:21.919164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:21.921108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:21.929890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:22.038734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1214/2439] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:21 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:22.042825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:22.053717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4cff5-4314-43c5-939d-e6dbdcea8234 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:22.083916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:22.110782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:22.111467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:22.111756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:22.112039 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:22.112809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:22.114745 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:22.207447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1226/2440] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:22 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:22.212150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:22.222548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050b7f0-a96b-4f7e-bed8-a15bacf3c945 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:22.228025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:22.228715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:22.228998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:22.229281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:22.230000 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:22.231646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:22.242990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1215/2441] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:22.247074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:22.257280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6743c5-7144-491e-b1e4-a8f21058fd3c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:22.261693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dba58-3ed1-4c1e-8431-cccbff902b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:22.262355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dba58-3ed1-4c1e-8431-cccbff902b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:22.262649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dba58-3ed1-4c1e-8431-cccbff902b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:22.262927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dba58-3ed1-4c1e-8431-cccbff902b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:22.263655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dba58-3ed1-4c1e-8431-cccbff902b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:22.265283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dba58-3ed1-4c1e-8431-cccbff902b5d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:22.276269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1227/2442] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:22.280284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:22.290759 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca949-e556-4726-ad6b-5bd517358c03 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:22.296372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:22.297295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:22.297584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:22.297863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] PATH_INFO: `/v3/projects/7e21654a69004b4dbeca5366892ddcb5/users/5fee1f51d2d445239b7aa2211d700e6a/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:22.312695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e21654a69004b4dbeca5366892ddcb5, user_id=5fee1f51d2d445239b7aa2211d700e6a, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:22.315381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:22.402317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1216/2443] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:22 2022] PUT /identity/v3/projects/7e21654a69004b4dbeca5366892ddcb5/users/5fee1f51d2d445239b7aa2211d700e6a/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:22.412385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c47d1-e9cc-4d0f-94ae-d07df2a613b7 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:22.413194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c47d1-e9cc-4d0f-94ae-d07df2a613b7 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:22.413476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c47d1-e9cc-4d0f-94ae-d07df2a613b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:22.438856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f41c47d1-e9cc-4d0f-94ae-d07df2a613b7 None None] MFA Rules not processed for user `5fee1f51d2d445239b7aa2211d700e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:22.440056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f41c47d1-e9cc-4d0f-94ae-d07df2a613b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:22.506668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1228/2444] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:22.518565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:22.520639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:22.594149 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7a405-c095-414b-905e-c4729c86e0b4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:22.599631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44ff0b-eb59-4a6c-ba63-b284d7308dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:22.600294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ff0b-eb59-4a6c-ba63-b284d7308dbc service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:22.600621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ff0b-eb59-4a6c-ba63-b284d7308dbc service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:22.600900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ff0b-eb59-4a6c-ba63-b284d7308dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:22.608849 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44ff0b-eb59-4a6c-ba63-b284d7308dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:22.610461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44ff0b-eb59-4a6c-ba63-b284d7308dbc service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:22.619224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1217/2445] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:25.983307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c47d1-e9cc-4d0f-94ae-d07df2a613b7 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:26.052009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c47d1-e9cc-4d0f-94ae-d07df2a613b7 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:26.052864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:26.053746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:26.054031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:26.054307 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] PATH_INFO: `/v3/users/5fee1f51d2d445239b7aa2211d700e6a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:26.055535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fee1f51d2d445239b7aa2211d700e6a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:26.057505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:26.247295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:26.249758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fee1f51d2d445239b7aa2211d700e6a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:26.250408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fee1f51d2d445239b7aa2211d700e6a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:26.251070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fee1f51d2d445239b7aa2211d700e6a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:26.257778 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fee1f51d2d445239b7aa2211d700e6a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:26.336858 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fee1f51d2d445239b7aa2211d700e6a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:26.354614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1229/2446] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:25 2022] DELETE /identity/v3/users/5fee1f51d2d445239b7aa2211d700e6a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:26.359259 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:26.359729 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:26.359852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:26.376617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:26.377828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:26.437902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1218/2447] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:26.448644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:26.450053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:26.506888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c1e77-ebc3-4c2b-9e38-6c8e3676f4dd admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:26.510480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed11bed-2aa8-4018-923e-b055f9a707e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:26.511136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed11bed-2aa8-4018-923e-b055f9a707e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:26.511422 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed11bed-2aa8-4018-923e-b055f9a707e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:26.511701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed11bed-2aa8-4018-923e-b055f9a707e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:26.522511 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed11bed-2aa8-4018-923e-b055f9a707e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:26.523957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed11bed-2aa8-4018-923e-b055f9a707e3 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:26.534016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1230/2448] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:26.951671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:26.953164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:26.965240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762ebd2-3269-491f-bbdf-c7f6e39e41f2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:26.969870 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:26.970810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:26.971087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:26.971367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] PATH_INFO: `/v3/projects/7e21654a69004b4dbeca5366892ddcb5` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:26.972641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e21654a69004b4dbeca5366892ddcb5)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:26.974558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:27.078355 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:27.087508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e21654a69004b4dbeca5366892ddcb5', 'request_id': 'req-a63c916f-15f3-4011-a399-5063f69ae012'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:27.092934 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e21654a69004b4dbeca5366892ddcb5', 'request_id': 'req-a63c916f-15f3-4011-a399-5063f69ae012'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:27.099153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e21654a69004b4dbeca5366892ddcb5', 'request_id': 'req-a63c916f-15f3-4011-a399-5063f69ae012'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:27.184057 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] The token cache is being invalidate because project 7e21654a69004b4dbeca5366892ddcb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:27.184350 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:27.184997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a63c916f-15f3-4011-a399-5063f69ae012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:27.186539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1219/2449] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:26 2022] DELETE /identity/v3/projects/7e21654a69004b4dbeca5366892ddcb5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:27.191965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfe063-4b4f-4421-874a-61c022462bd0 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.192419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfe063-4b4f-4421-874a-61c022462bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.192557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfe063-4b4f-4421-874a-61c022462bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.209737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d0bfe063-4b4f-4421-874a-61c022462bd0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:27.210930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d0bfe063-4b4f-4421-874a-61c022462bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:27.268979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1231/2450] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:27.277569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d49bd-97d9-4497-b55b-2ef8eeacc7b4 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.277860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d49bd-97d9-4497-b55b-2ef8eeacc7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.278138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d49bd-97d9-4497-b55b-2ef8eeacc7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.297042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-db3d49bd-97d9-4497-b55b-2ef8eeacc7b4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:27.298246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-db3d49bd-97d9-4497-b55b-2ef8eeacc7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:27.306804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1220/2451] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:27.311054 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfe063-4b4f-4421-874a-61c022462bd0 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:27.320990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfe063-4b4f-4421-874a-61c022462bd0 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:27.325505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:27.326228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.326513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.326788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.327545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:56:27.327849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:27.329750 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:27.341702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1232/2452] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:56:27.351836 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d49bd-97d9-4497-b55b-2ef8eeacc7b4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:27.360998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d49bd-97d9-4497-b55b-2ef8eeacc7b4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:27.366170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:27.366812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.367098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.367376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.368076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:27.370087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:27.450671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1221/2453] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:27 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:27.454920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:27.464844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d193c-1705-4ba0-bc74-a40b1bba1ae1 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:27.512784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:27.522188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:27.522852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.523135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.523412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.524111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:27.526119 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:27.654270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1233/2454] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:27 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:27.660061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:27.677042 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319ff69-3c31-437d-8271-85b7061deaf6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:27.685292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:27.686450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.686962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.687468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.688975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:27.692627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:27.714874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1222/2455] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:27.716454 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:27.727197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5884d-6685-4b30-95ba-8c959928617c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:27.732037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f444b15-1cac-438f-b04a-a739595e0c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:27.732898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f444b15-1cac-438f-b04a-a739595e0c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.733193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f444b15-1cac-438f-b04a-a739595e0c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.733594 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f444b15-1cac-438f-b04a-a739595e0c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.734332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f444b15-1cac-438f-b04a-a739595e0c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:27.735929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f444b15-1cac-438f-b04a-a739595e0c48 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:27.751302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1234/2456] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:27.752579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:27.762808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959d17-26ae-43db-b078-5c4f3a3229ed admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:27.767386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:27.768274 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.768606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.768887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] PATH_INFO: `/v3/projects/f3a6383b246349f893c646887e558610/users/56520757c79e42e5b857e4fc450d1db5/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.784039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a6383b246349f893c646887e558610, user_id=56520757c79e42e5b857e4fc450d1db5, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:27.786581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:27.889451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1223/2457] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:27 2022] PUT /identity/v3/projects/f3a6383b246349f893c646887e558610/users/56520757c79e42e5b857e4fc450d1db5/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:27.899686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4308e-6700-46d4-9204-01e8c4eaa1cf None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:27.899979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4308e-6700-46d4-9204-01e8c4eaa1cf None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:27.900260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4308e-6700-46d4-9204-01e8c4eaa1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:27.926611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-99d4308e-6700-46d4-9204-01e8c4eaa1cf None None] MFA Rules not processed for user `56520757c79e42e5b857e4fc450d1db5`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:27.928409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-99d4308e-6700-46d4-9204-01e8c4eaa1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:27.991959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1235/2458] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:27.996534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:28.057942 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06385527-9d9d-4852-b71d-04698fbff0f5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:28.062525 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:28.063232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.063519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.063793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.064511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:28.066665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:28.194258 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1224/2459] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:27 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:28.198329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4308e-6700-46d4-9204-01e8c4eaa1cf None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:28.208808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4308e-6700-46d4-9204-01e8c4eaa1cf None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:28.262449 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:28.263105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.263386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.263658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.264331 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:28.266405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:28.353425 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1236/2460] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:28 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:28.363231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:28.374182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e22842-55c4-4491-aefc-42827f86c4b3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:28.379187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:28.379869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.380147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.380453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.381212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:28.383219 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:28.395506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1225/2461] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:28.399326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:28.409548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909afb7a-0420-4bd3-b480-8f2b346a636f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:28.416056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:28.417017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.417837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.417837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] PATH_INFO: `/v3/projects/c84e8d269218448fbe1167a8d2f79214/users/468297eaafe248cf95bd38b354c96478/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.435956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84e8d269218448fbe1167a8d2f79214, user_id=468297eaafe248cf95bd38b354c96478, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:28.438695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:28.518681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1237/2462] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:28 2022] PUT /identity/v3/projects/c84e8d269218448fbe1167a8d2f79214/users/468297eaafe248cf95bd38b354c96478/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:28.523281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:28.534101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d6f98-6a62-474f-9fd5-8fba20588b26 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:28.570953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:28.599907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f6b87-4560-4bf4-9c3a-c1c45239564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:28.600914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f6b87-4560-4bf4-9c3a-c1c45239564d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.601016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f6b87-4560-4bf4-9c3a-c1c45239564d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.601222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f6b87-4560-4bf4-9c3a-c1c45239564d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.601950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f6b87-4560-4bf4-9c3a-c1c45239564d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:28.603704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f6b87-4560-4bf4-9c3a-c1c45239564d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:28.618303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1226/2463] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:28.619614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:28.630121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec1ae5-0096-4a21-bf99-733f0b3e882e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:28.635541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:28.636781 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.637219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.637704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] PATH_INFO: `/v3/projects/c84e8d269218448fbe1167a8d2f79214/users/468297eaafe248cf95bd38b354c96478/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.639718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84e8d269218448fbe1167a8d2f79214, user_id=468297eaafe248cf95bd38b354c96478, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:28.642641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:28.654803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1238/2464] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:28 2022] PUT /identity/v3/projects/c84e8d269218448fbe1167a8d2f79214/users/468297eaafe248cf95bd38b354c96478/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:28.665257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.665646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.666023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.699443 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] MFA Rules not processed for user `468297eaafe248cf95bd38b354c96478`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:28.701092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:28.779973 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1227/2465] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:28.792401 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:28.795816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:28.814539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:28.876006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6a029-9957-487e-84aa-b1cd9e142dde admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:28.880191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:28.881001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:28.881805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:28.882075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:28.892690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:28.894165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:28.906228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1239/2466] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:29.041951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:29.043675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:29.058839 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318bac-f492-4f80-b8b2-5a902fb118a9 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:29.062621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f632a-3e43-457b-880b-4ce0bb2a8c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:29.063287 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f632a-3e43-457b-880b-4ce0bb2a8c22 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:29.063570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f632a-3e43-457b-880b-4ce0bb2a8c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:29.063846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f632a-3e43-457b-880b-4ce0bb2a8c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:29.145060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f632a-3e43-457b-880b-4ce0bb2a8c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:29.148070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f632a-3e43-457b-880b-4ce0bb2a8c22 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:29.161367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1228/2467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:34.478550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:34.540064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63db19-727c-45b5-8bea-564e106cc9c9 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:34.544629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:34.545491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:34.545773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:34.546049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] PATH_INFO: `/v3/users/56520757c79e42e5b857e4fc450d1db5` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:34.547222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=56520757c79e42e5b857e4fc450d1db5)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:34.549227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:34.731115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56520757c79e42e5b857e4fc450d1db5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:34.731806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56520757c79e42e5b857e4fc450d1db5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:34.732402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56520757c79e42e5b857e4fc450d1db5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:34.739219 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56520757c79e42e5b857e4fc450d1db5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:34.831676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56520757c79e42e5b857e4fc450d1db5'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:34.853693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1240/2468] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:34 2022] DELETE /identity/v3/users/56520757c79e42e5b857e4fc450d1db5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:34.858883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:34.859319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:34.859319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:34.876271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:34.877565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:34.931374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:34.945313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1229/2469] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:34.953333 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:34.954839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:34.996140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:35.025375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da00db7-b90e-4123-bb3e-f6c5bd4716ff admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:35.029541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:35.030195 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:35.030492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:35.030841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:35.041290 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:35.042733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:35.052450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1241/2470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:35.459911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:35.461504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:35.474394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092c9de-8a78-4c15-8940-9ed7a58aef67 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:35.479700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:35.487370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:35.487370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:35.487370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] PATH_INFO: `/v3/projects/f3a6383b246349f893c646887e558610` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:35.487370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a6383b246349f893c646887e558610)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:35.487370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:35.608975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a6383b246349f893c646887e558610', 'request_id': 'req-587dc3f7-397f-4f77-b214-bae562cba456'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.614356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a6383b246349f893c646887e558610', 'request_id': 'req-587dc3f7-397f-4f77-b214-bae562cba456'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.620949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a6383b246349f893c646887e558610', 'request_id': 'req-587dc3f7-397f-4f77-b214-bae562cba456'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.714627 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] The token cache is being invalidate because project f3a6383b246349f893c646887e558610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:35.714919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.715537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.717141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1230/2471] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:35 2022] DELETE /identity/v3/projects/f3a6383b246349f893c646887e558610 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:35.721189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:35.730719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f2d38-5605-497e-81af-1e0a2ef1d903 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:35.794247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:35.795033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:35.795316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:35.795590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] PATH_INFO: `/v3/users/468297eaafe248cf95bd38b354c96478` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:35.796953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=468297eaafe248cf95bd38b354c96478)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:35.798959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:35.988008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468297eaafe248cf95bd38b354c96478'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.988788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468297eaafe248cf95bd38b354c96478'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.989386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468297eaafe248cf95bd38b354c96478'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:35.996821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468297eaafe248cf95bd38b354c96478'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:36.077835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468297eaafe248cf95bd38b354c96478'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:36.095521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1242/2472] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:35 2022] DELETE /identity/v3/users/468297eaafe248cf95bd38b354c96478 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:36.378158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:36.379748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:36.440380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dc3f7-397f-4f77-b214-bae562cba456 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:36.492969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7ec6a-262b-4e19-91b4-2dfe140d9f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:36.493619 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7ec6a-262b-4e19-91b4-2dfe140d9f37 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:36.493911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7ec6a-262b-4e19-91b4-2dfe140d9f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:36.494183 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7ec6a-262b-4e19-91b4-2dfe140d9f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:36.495946 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-40d7ec6a-262b-4e19-91b4-2dfe140d9f37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:36.531590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:36.533222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:36.542369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:36.563455 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7ec6a-262b-4e19-91b4-2dfe140d9f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:36.564877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7ec6a-262b-4e19-91b4-2dfe140d9f37 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:36.575252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1231/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:36.600535 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:36.605215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c3c37-27f0-419c-9afc-403ef515aebe admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:36.609806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:36.610685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:36.610970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:36.611244 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] PATH_INFO: `/v3/projects/c84e8d269218448fbe1167a8d2f79214` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:36.612487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84e8d269218448fbe1167a8d2f79214)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:36.614412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:36.718841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84e8d269218448fbe1167a8d2f79214', 'request_id': 'req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:36.725625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84e8d269218448fbe1167a8d2f79214', 'request_id': 'req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:36.786343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84e8d269218448fbe1167a8d2f79214', 'request_id': 'req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:36.800648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] The token cache is being invalidate because project c84e8d269218448fbe1167a8d2f79214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:36.801268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:36.801969 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-fc77db82-5b9f-4aa0-bb43-988efff3d8f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:36.803590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1243/2474] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:36 2022] DELETE /identity/v3/projects/c84e8d269218448fbe1167a8d2f79214 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:36.808403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0637b-3d80-465f-9131-cbac7ea9ca1b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:36.808737 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0637b-3d80-465f-9131-cbac7ea9ca1b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:36.809017 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0637b-3d80-465f-9131-cbac7ea9ca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:36.827268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-62f0637b-3d80-465f-9131-cbac7ea9ca1b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:36.828508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-62f0637b-3d80-465f-9131-cbac7ea9ca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:36.889013 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1232/2475] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:36.898657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7bf35-205c-496b-b908-804a6c495e7d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:36.898949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7bf35-205c-496b-b908-804a6c495e7d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:36.899232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7bf35-205c-496b-b908-804a6c495e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:36.918321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e2d7bf35-205c-496b-b908-804a6c495e7d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:36.919807 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e2d7bf35-205c-496b-b908-804a6c495e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:36.927941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1244/2476] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:36.932187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0637b-3d80-465f-9131-cbac7ea9ca1b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:36.942016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0637b-3d80-465f-9131-cbac7ea9ca1b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:36.947172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:36.947893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:36.948243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:36.948617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:36.949412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:56:36.949786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:36.951479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:36.966694 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1233/2477] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:56:36.967963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7bf35-205c-496b-b908-804a6c495e7d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:36.977350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7bf35-205c-496b-b908-804a6c495e7d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:36.982010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:36.982692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:36.982981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:36.983386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:36.984084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:36.986009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:37.128558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1245/2478] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:36 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:37.133276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:37.143616 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf971ac1-a831-4b87-9ec3-bdb355f71531 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:37.202653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:37.203325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:37.203613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:37.203889 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:37.204626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:37.206553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:37.327583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1234/2479] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:37 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:37.329177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:37.341640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81234c71-6ccf-4675-aa92-b990997e324d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:37.346492 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:37.347278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:37.347555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:37.347828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:37.348619 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:37.350208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:37.361002 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1246/2480] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:37.365584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:37.375975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42553ed1-b5a9-4935-86f8-07ab51e7e19f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:37.380763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e797c48-340c-4f1b-b937-afcd6c551a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:37.385235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797c48-340c-4f1b-b937-afcd6c551a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:37.385235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797c48-340c-4f1b-b937-afcd6c551a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:37.385235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797c48-340c-4f1b-b937-afcd6c551a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:37.385235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e797c48-340c-4f1b-b937-afcd6c551a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:37.385235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e797c48-340c-4f1b-b937-afcd6c551a52 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:37.395657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1235/2481] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:37.399766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:37.409773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96823000-4ff9-4dd4-996a-1f3e79774c21 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:37.414472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:37.415343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:37.415626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:37.415902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] PATH_INFO: `/v3/projects/661d1e6677234535a683b3d59315f699/users/485cc64d46474136b121cf35f761e346/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:37.430696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] RBAC: Authorizing `identity:create_grant(project_id=661d1e6677234535a683b3d59315f699, user_id=485cc64d46474136b121cf35f761e346, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:37.433393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:37.537636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1247/2482] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:37 2022] PUT /identity/v3/projects/661d1e6677234535a683b3d59315f699/users/485cc64d46474136b121cf35f761e346/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:37.547898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6f6b3-0010-4919-ab8d-8202315de7d5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:37.548195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6f6b3-0010-4919-ab8d-8202315de7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:37.548502 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6f6b3-0010-4919-ab8d-8202315de7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:37.579904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-fba6f6b3-0010-4919-ab8d-8202315de7d5 None None] MFA Rules not processed for user `485cc64d46474136b121cf35f761e346`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:37.581252 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-fba6f6b3-0010-4919-ab8d-8202315de7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:37.633497 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:37.666768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1236/2483] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:37.669439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:37.732052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b7cc0-65d3-43f7-9dc2-c7263ca7083a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:37.736840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:37.737502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:37.737788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:37.738068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:37.738762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:37.741388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:37.820277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1248/2484] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:37 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:37.824267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6f6b3-0010-4919-ab8d-8202315de7d5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:37.836028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6f6b3-0010-4919-ab8d-8202315de7d5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:37.901438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:37.901438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:37.901438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:37.901438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:37.901918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:37.902359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:37.993235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1237/2485] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:37 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:37.993651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:38.005184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab998b9-6511-49ab-962a-3a21c04e9392 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:38.010320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:38.011261 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.011727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.012174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.013498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:38.015293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:38.017329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:38.029956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1249/2486] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:38.037753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:38.050960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f27156-cc28-420b-8f0c-aad266b37fd4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:38.058396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:38.059938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.060382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.060872 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] PATH_INFO: `/v3/projects/9f76c4a540b7472fb488c3fa00f566b8/users/f82c4126c06b4be283a06a5986c0d1bf/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.080997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f76c4a540b7472fb488c3fa00f566b8, user_id=f82c4126c06b4be283a06a5986c0d1bf, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:38.085401 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:38.174815 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1238/2487] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:38 2022] PUT /identity/v3/projects/9f76c4a540b7472fb488c3fa00f566b8/users/f82c4126c06b4be283a06a5986c0d1bf/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:38.177757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:38.197506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca118015-595c-42c7-9b72-ac3837fe6750 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:38.269533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f5fe3-9d41-4982-85db-b2dddcb601ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:38.270245 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f5fe3-9d41-4982-85db-b2dddcb601ed admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.271101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f5fe3-9d41-4982-85db-b2dddcb601ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.271101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f5fe3-9d41-4982-85db-b2dddcb601ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.272279 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f5fe3-9d41-4982-85db-b2dddcb601ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:38.274101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f5fe3-9d41-4982-85db-b2dddcb601ed admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:38.284822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1250/2488] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:38.289570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:38.299930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67ee77-96fe-42c1-9842-5b0759553865 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:38.305452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:38.306161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.306445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.306727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] PATH_INFO: `/v3/projects/9f76c4a540b7472fb488c3fa00f566b8/users/f82c4126c06b4be283a06a5986c0d1bf/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.309751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f76c4a540b7472fb488c3fa00f566b8, user_id=f82c4126c06b4be283a06a5986c0d1bf, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:38.312827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:38.324705 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1239/2489] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:38 2022] PUT /identity/v3/projects/9f76c4a540b7472fb488c3fa00f566b8/users/f82c4126c06b4be283a06a5986c0d1bf/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:38.339136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cf573-a842-4c88-97be-e00cdd16005e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.339429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cf573-a842-4c88-97be-e00cdd16005e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.339715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cf573-a842-4c88-97be-e00cdd16005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.369079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-726cf573-a842-4c88-97be-e00cdd16005e None None] MFA Rules not processed for user `f82c4126c06b4be283a06a5986c0d1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:38.369912 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-726cf573-a842-4c88-97be-e00cdd16005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:38.452558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1251/2490] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:38.460475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:38.462984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:38.525441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9871fe-3e0f-4df6-ab15-566fbc6bbe07 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:38.530837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09245e-075e-4fe7-8b05-3b4465ffc956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:38.531585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09245e-075e-4fe7-8b05-3b4465ffc956 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.531924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09245e-075e-4fe7-8b05-3b4465ffc956 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.532254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09245e-075e-4fe7-8b05-3b4465ffc956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.545546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09245e-075e-4fe7-8b05-3b4465ffc956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:38.547100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09245e-075e-4fe7-8b05-3b4465ffc956 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:38.559313 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1240/2491] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:38.633734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cf573-a842-4c88-97be-e00cdd16005e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:38.703500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cf573-a842-4c88-97be-e00cdd16005e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:38.709630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:38.710700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.710871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.711154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] PATH_INFO: `/v3/users/485cc64d46474136b121cf35f761e346` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.712495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=485cc64d46474136b121cf35f761e346)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:38.714460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:38.739508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:38.873653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485cc64d46474136b121cf35f761e346'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:38.874300 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485cc64d46474136b121cf35f761e346'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:38.877439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485cc64d46474136b121cf35f761e346'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:38.882361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485cc64d46474136b121cf35f761e346'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:38.952386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485cc64d46474136b121cf35f761e346'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:38.971959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1252/2492] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:38 2022] DELETE /identity/v3/users/485cc64d46474136b121cf35f761e346 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:38.977061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:38.977544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:38.977835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:38.996055 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:38.997283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:39.027988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:39.067516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1241/2493] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:39.084842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:39.087463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:39.185021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d982a-ccf8-48a4-95a7-1e6efba9e07e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:39.190303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:39.191125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:39.192719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:39.193804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:39.204984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:39.206729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:39.219689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1253/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:56:39.811639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:39.813847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:39.825542 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a872be-93fa-44c0-8282-3e1d545320da None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:39.830379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:39.831290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:39.831569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:39.831855 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] PATH_INFO: `/v3/projects/661d1e6677234535a683b3d59315f699` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:39.833093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=661d1e6677234535a683b3d59315f699)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:39.835046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:39.942715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661d1e6677234535a683b3d59315f699', 'request_id': 'req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:39.959465 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661d1e6677234535a683b3d59315f699', 'request_id': 'req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:39.967796 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661d1e6677234535a683b3d59315f699', 'request_id': 'req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.043662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] The token cache is being invalidate because project 661d1e6677234535a683b3d59315f699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:40.048769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.048769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.048769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1242/2495] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:39 2022] DELETE /identity/v3/projects/661d1e6677234535a683b3d59315f699 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:40.050360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:40.061709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ef72c-e9bb-4c7d-b2ef-f5abe0ce464f service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:40.136297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:40.137352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:40.137772 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:40.138240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] PATH_INFO: `/v3/users/f82c4126c06b4be283a06a5986c0d1bf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:40.139694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82c4126c06b4be283a06a5986c0d1bf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:40.141782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:40.352125 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:40.357394 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82c4126c06b4be283a06a5986c0d1bf'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.358038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82c4126c06b4be283a06a5986c0d1bf'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.358638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82c4126c06b4be283a06a5986c0d1bf'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.366015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82c4126c06b4be283a06a5986c0d1bf'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.459922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82c4126c06b4be283a06a5986c0d1bf'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.464373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:40.469686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:40.483014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1254/2496] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:40 2022] DELETE /identity/v3/users/f82c4126c06b4be283a06a5986c0d1bf => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:40.541947 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1c1a6-b951-47ca-a0ca-a40919ebd89a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:40.596305 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:40.603767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:40.604418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:40.604729 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:40.605001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:40.607227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:40.620079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:40.626533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:40.628403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:40.630003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:40.635734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1243/2497] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:56:40.718340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34730ba-28f0-48aa-b4ff-3f00d36e9ccf admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:40.723180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:40.724058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:40.724344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:40.724713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] PATH_INFO: `/v3/users/3077a7934f9c4407a70a8790b8da9cde` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:40.725922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3077a7934f9c4407a70a8790b8da9cde)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:40.727866 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:40.959222 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:40.960755 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:40.983327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63b906-4fee-4808-8933-296e6a444b35 service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:40.983327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:40.983327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:40.983327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:40.983746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] PATH_INFO: `/v3/projects/9f76c4a540b7472fb488c3fa00f566b8` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:40.984083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f76c4a540b7472fb488c3fa00f566b8)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:40.986679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:40.989666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3077a7934f9c4407a70a8790b8da9cde'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.990952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3077a7934f9c4407a70a8790b8da9cde'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.991527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3077a7934f9c4407a70a8790b8da9cde'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:40.999046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3077a7934f9c4407a70a8790b8da9cde'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.043521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d726090f-023e-4677-8b54-ac296fcda8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3077a7934f9c4407a70a8790b8da9cde'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.061737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1255/2498] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:40 2022] DELETE /identity/v3/users/3077a7934f9c4407a70a8790b8da9cde => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:41.088507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f76c4a540b7472fb488c3fa00f566b8', 'request_id': 'req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.097800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f76c4a540b7472fb488c3fa00f566b8', 'request_id': 'req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.109241 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f76c4a540b7472fb488c3fa00f566b8', 'request_id': 'req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.185631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] The token cache is being invalidate because project 9f76c4a540b7472fb488c3fa00f566b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:41.185631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.185978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-bcbc79e7-7b3b-4341-9ca8-31d6b9c9491d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.187544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1244/2499] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:40 2022] DELETE /identity/v3/projects/9f76c4a540b7472fb488c3fa00f566b8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:41.192696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83275e2-512c-465e-b4e7-3b29893e9596 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.193043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83275e2-512c-465e-b4e7-3b29893e9596 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.193905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83275e2-512c-465e-b4e7-3b29893e9596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.211939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d83275e2-512c-465e-b4e7-3b29893e9596 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:41.213114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d83275e2-512c-465e-b4e7-3b29893e9596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:41.275836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1256/2500] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:41.283803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.284090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.284367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.303242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:41.308314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:41.312389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1245/2501] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:41.316971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83275e2-512c-465e-b4e7-3b29893e9596 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.326463 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83275e2-512c-465e-b4e7-3b29893e9596 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.330968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.331629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.331916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.332196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.332968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:56:41.333509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.335304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.339637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.341098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:41.346526 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:41.351838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1257/2502] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:56:41.355117 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.358828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1b1da-b99e-4113-8fb6-19781c2cc8c2 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.364864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.366614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.366777 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.366964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ca925-ea7d-47fa-bc09-89f8cbf7ac34 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.367072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] PATH_INFO: `/v3/projects/93918dc6b6f445d18a3ecb003524152d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.368253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=93918dc6b6f445d18a3ecb003524152d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.370512 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.372708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.373755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.374048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.374365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.375053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.377092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.505818 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1258/2503] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/projects => generated 389 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:56:41.509249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.545046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93918dc6b6f445d18a3ecb003524152d', 'request_id': 'req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.552172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93918dc6b6f445d18a3ecb003524152d', 'request_id': 'req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.559088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93918dc6b6f445d18a3ecb003524152d', 'request_id': 'req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.574900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5174b2-ee4b-4320-82f4-dfa5b652b1ee admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.630628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.631299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.631587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.631874 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.632700 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.634666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.652702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] The token cache is being invalidate because project 93918dc6b6f445d18a3ecb003524152d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:41.653076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.653727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3b0570c0-a536-41f6-bdad-ddec8d1f8cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:41.655337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1246/2504] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:41 2022] DELETE /identity/v3/projects/93918dc6b6f445d18a3ecb003524152d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:41.660065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defee64-3f7f-4fdc-8e1e-26e2d30fb40b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.660360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defee64-3f7f-4fdc-8e1e-26e2d30fb40b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.660680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defee64-3f7f-4fdc-8e1e-26e2d30fb40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.680282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1defee64-3f7f-4fdc-8e1e-26e2d30fb40b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:41.681518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1defee64-3f7f-4fdc-8e1e-26e2d30fb40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:41.689472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1247/2505] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:41.697726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da50bc-c813-46c8-9801-98a66de0786d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.698018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da50bc-c813-46c8-9801-98a66de0786d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.698294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da50bc-c813-46c8-9801-98a66de0786d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.715943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-22da50bc-c813-46c8-9801-98a66de0786d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:41.717136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-22da50bc-c813-46c8-9801-98a66de0786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:41.725430 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1248/2506] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:41.729302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da50bc-c813-46c8-9801-98a66de0786d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.737926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da50bc-c813-46c8-9801-98a66de0786d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.741644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1259/2507] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/users => generated 295 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:56:41.742636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.742867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.743511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.743910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.744137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.744253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:41.744892 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:56:41.745200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.746788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.755801 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842466-948e-4da8-a213-b7749cfac6a5 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.759507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1249/2508] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:56:41.760603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.761270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.761554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.761867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.762695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.763708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.764295 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.774315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c82aa4-f04f-471b-b58f-098a4116b4e5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.780105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1260/2509] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:41.780223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.780223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.780424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.780656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.781020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.781374 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.783261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.791379 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810fc1e-94eb-4166-9325-c99f0c694c36 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.796099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.796829 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.801685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.801685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.801685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.801685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.812084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1261/2510] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:41.816174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.826720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a4110-0403-4dbb-8121-ca49bc6c2fe0 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:41.831293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5861d3-97c5-4ead-92cc-81fe9b85f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:41.832152 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5861d3-97c5-4ead-92cc-81fe9b85f8f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.832460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5861d3-97c5-4ead-92cc-81fe9b85f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.832731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5861d3-97c5-4ead-92cc-81fe9b85f8f0 admin admin] PATH_INFO: `/v3/projects/dcec0de2c3c3477fa85313a1673afa22/users/71635f8057694599bf1b4460df4fd413/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:41.847370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5861d3-97c5-4ead-92cc-81fe9b85f8f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcec0de2c3c3477fa85313a1673afa22, user_id=71635f8057694599bf1b4460df4fd413, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:41.850098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5861d3-97c5-4ead-92cc-81fe9b85f8f0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:41.936193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:41.942869 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1250/2511] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:41.947510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:41.987022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1262/2512] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:41 2022] PUT /identity/v3/projects/dcec0de2c3c3477fa85313a1673afa22/users/71635f8057694599bf1b4460df4fd413/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:41.997798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa9204-0d69-44ce-ad97-b54a183dac37 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:41.998085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa9204-0d69-44ce-ad97-b54a183dac37 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:41.998366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa9204-0d69-44ce-ad97-b54a183dac37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.008234 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac789-816e-47e6-b02b-94e7f58bb3ea admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.028118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-caaa9204-0d69-44ce-ad97-b54a183dac37 None None] MFA Rules not processed for user `71635f8057694599bf1b4460df4fd413`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:42.030157 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-caaa9204-0d69-44ce-ad97-b54a183dac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:42.063533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.064202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.064514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.064821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.065511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.066634 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:42.067686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.098585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1263/2513] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:42.105641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa9204-0d69-44ce-ad97-b54a183dac37 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.117094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa9204-0d69-44ce-ad97-b54a183dac37 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.121633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.122316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.122605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.122887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.123592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.125609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.162731 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1251/2514] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:41 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:56:42.165304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.174680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0b89b-ad16-48b6-8fcf-3742c82b678a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.179775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.180487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.181003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.181284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.182025 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.183600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.194418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1252/2515] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:42.198198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.207527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba07cba-fd83-4eb0-8516-8ca17710de9d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.212175 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.212821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.213102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.213380 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.214069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.215631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.226898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1253/2516] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:42.230360 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.239978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f61a22-773a-40a5-bdc9-74ae25e6f6eb admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.245351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe758e0-6070-473a-9c0c-a796461971ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.246787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe758e0-6070-473a-9c0c-a796461971ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.246787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe758e0-6070-473a-9c0c-a796461971ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.246787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe758e0-6070-473a-9c0c-a796461971ca admin admin] PATH_INFO: `/v3/projects/32674399b00b4253a370f05ad7297215/users/cdd216ed3f474929b7f9c8f8f79df617/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.253351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1264/2517] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:42 2022] POST /identity/v3/projects => generated 389 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:42.254348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.261312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe758e0-6070-473a-9c0c-a796461971ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=32674399b00b4253a370f05ad7297215, user_id=cdd216ed3f474929b7f9c8f8f79df617, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.263993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe758e0-6070-473a-9c0c-a796461971ca admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.321003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0cd0-d3a4-457c-86f3-e19606703fc6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.325679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.326477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.327198 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.327944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.329643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.331720 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.362064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1254/2518] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:42 2022] PUT /identity/v3/projects/32674399b00b4253a370f05ad7297215/users/cdd216ed3f474929b7f9c8f8f79df617/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:42.372904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b2466-eece-4e62-9efb-0e8f8d8a9d4b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.373195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b2466-eece-4e62-9efb-0e8f8d8a9d4b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.373472 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b2466-eece-4e62-9efb-0e8f8d8a9d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.399171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-667b2466-eece-4e62-9efb-0e8f8d8a9d4b None None] MFA Rules not processed for user `cdd216ed3f474929b7f9c8f8f79df617`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:42.400384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-667b2466-eece-4e62-9efb-0e8f8d8a9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:42.413362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1265/2519] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:42 2022] POST /identity/v3/users => generated 301 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:42.417502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.426997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8378642-224d-422c-a9c1-94fc85eb4ff5 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.441638 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:42.466293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1255/2520] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:42.470783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b2466-eece-4e62-9efb-0e8f8d8a9d4b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.477161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:42.488197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.488898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.489616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.489962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.490705 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.492303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.502115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1266/2521] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:42.506230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.515639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7741529-717c-46a9-8d3d-dfb05372bb2c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.520418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.521824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.522115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.522393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] PATH_INFO: `/v3/projects/f679f8f994de473bb27a309d706db297/users/7c6d8e57a1d341489d23d0a1d1d576d8/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.535011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b2466-eece-4e62-9efb-0e8f8d8a9d4b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.537785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f679f8f994de473bb27a309d706db297, user_id=7c6d8e57a1d341489d23d0a1d1d576d8, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.542832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.544399 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.545048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.545338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.545619 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.546326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.548270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.629641 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1267/2522] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:42 2022] PUT /identity/v3/projects/f679f8f994de473bb27a309d706db297/users/7c6d8e57a1d341489d23d0a1d1d576d8/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:42.633452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.663745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1256/2523] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:42 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:56:42.667714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.699521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b1ae2-7031-4d20-b0dd-932643ae6e4a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.729053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74510367-3003-47e0-9402-6c8b79f29977 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.733927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.734594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.734941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.735247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.735937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.737898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.758531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.759232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.759515 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.759790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.760550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.762167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.772478 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1268/2524] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:42.776563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.786156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9dcc8-fc49-461a-950b-3e2267122c31 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.790726 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370737e-574b-4cb6-b23f-c3f03af33676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.791553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370737e-574b-4cb6-b23f-c3f03af33676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.791837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370737e-574b-4cb6-b23f-c3f03af33676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.792113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370737e-574b-4cb6-b23f-c3f03af33676 admin admin] PATH_INFO: `/v3/projects/f679f8f994de473bb27a309d706db297/users/7c6d8e57a1d341489d23d0a1d1d576d8/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.794085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370737e-574b-4cb6-b23f-c3f03af33676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f679f8f994de473bb27a309d706db297, user_id=7c6d8e57a1d341489d23d0a1d1d576d8, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.796783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370737e-574b-4cb6-b23f-c3f03af33676 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.806779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1269/2525] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:42 2022] PUT /identity/v3/projects/f679f8f994de473bb27a309d706db297/users/7c6d8e57a1d341489d23d0a1d1d576d8/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:42.870151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.870442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.870729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.882510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1257/2526] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:42 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:56:42.884228 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.893517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233391-6454-4abe-92c8-c03ecea33cb4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.899915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] MFA Rules not processed for user `7c6d8e57a1d341489d23d0a1d1d576d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:42.901318 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:42.943934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:42.946160 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:42.952898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.958024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.958024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.958024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:42.958024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:42.958024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:42.966746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1258/2527] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:42.970787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.980392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06010e2a-0363-48c3-9596-e0129f8bc0a3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:42.981540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1270/2528] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:42.986217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:42.986406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:42.987298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:42.987581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:42.987690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:42.987857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] PATH_INFO: `/v3/projects/bb6a8e94e7674bbf81b795fe75be128d/users/bdacaaaa213148d8861ad17a90ca9353/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.003912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a8e94e7674bbf81b795fe75be128d, user_id=bdacaaaa213148d8861ad17a90ca9353, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.007638 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.043160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebbe0c-86a4-48fb-815b-d2d5a2d4122c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:43.046796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:43.047439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.047725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.048004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.057488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.058907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.068734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1271/2529] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:56:43.113460 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1259/2530] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:42 2022] PUT /identity/v3/projects/bb6a8e94e7674bbf81b795fe75be128d/users/bdacaaaa213148d8861ad17a90ca9353/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:43.117419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:43.130063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2322ec5-ad5f-4ea5-9e86-43256d39e78c service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:43.190058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:43.190729 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.191031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.191291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.192280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.194238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.200717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:43.205460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1272/2531] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:43.209784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:43.211333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09644550-2a85-4472-b9da-a5bd56ec9a69 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:43.215860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:43.216752 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.217035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.217311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] PATH_INFO: `/v3/users/71635f8057694599bf1b4460df4fd413` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.218817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71635f8057694599bf1b4460df4fd413)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.219616 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724b666-eb14-4345-b435-8d2f1f509642 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:43.220766 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.224263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40e640-e982-4558-bbf6-75086c0b765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:43.225220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40e640-e982-4558-bbf6-75086c0b765e admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.225510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40e640-e982-4558-bbf6-75086c0b765e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.225788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40e640-e982-4558-bbf6-75086c0b765e admin admin] PATH_INFO: `/v3/projects/bb6a8e94e7674bbf81b795fe75be128d/users/bdacaaaa213148d8861ad17a90ca9353/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.227794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40e640-e982-4558-bbf6-75086c0b765e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6a8e94e7674bbf81b795fe75be128d, user_id=bdacaaaa213148d8861ad17a90ca9353, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.230447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40e640-e982-4558-bbf6-75086c0b765e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.242009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1273/2532] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:43 2022] PUT /identity/v3/projects/bb6a8e94e7674bbf81b795fe75be128d/users/bdacaaaa213148d8861ad17a90ca9353/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:56:43.251991 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110881a6-f71c-4d8c-9fea-142ed9ff7fbf None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.252284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110881a6-f71c-4d8c-9fea-142ed9ff7fbf None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.252603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110881a6-f71c-4d8c-9fea-142ed9ff7fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.278276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-110881a6-f71c-4d8c-9fea-142ed9ff7fbf None None] MFA Rules not processed for user `bdacaaaa213148d8861ad17a90ca9353`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:43.279479 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-110881a6-f71c-4d8c-9fea-142ed9ff7fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:43.334230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:43.352534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1274/2533] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:43.363670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110881a6-f71c-4d8c-9fea-142ed9ff7fbf None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:43.417034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110881a6-f71c-4d8c-9fea-142ed9ff7fbf None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:43.420806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2837ef9-e0ec-469c-88b6-8a04181e5fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:43.421448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2837ef9-e0ec-469c-88b6-8a04181e5fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.421735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2837ef9-e0ec-469c-88b6-8a04181e5fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.422014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2837ef9-e0ec-469c-88b6-8a04181e5fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.429559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71635f8057694599bf1b4460df4fd413'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:43.433026 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2837ef9-e0ec-469c-88b6-8a04181e5fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.434503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2837ef9-e0ec-469c-88b6-8a04181e5fd3 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.443840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71635f8057694599bf1b4460df4fd413'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:43.444555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71635f8057694599bf1b4460df4fd413'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:43.444961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1275/2534] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:56:43.445195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71635f8057694599bf1b4460df4fd413'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:43.452147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71635f8057694599bf1b4460df4fd413'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:43.538302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1260/2535] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:43 2022] DELETE /identity/v3/users/71635f8057694599bf1b4460df4fd413 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:43.542900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.543201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.547533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.561620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:43.563080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:43.624107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1276/2536] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:43.633830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:43.635255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:43.672352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:43.673881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:43.694613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca413e-c5f7-4aca-a595-f7c0af7109b6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:43.698334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:43.698976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.699310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.699590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.708596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.710847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.719442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1261/2537] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:43.734289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25604d0e-b5cb-4099-8e29-34bd9a490f19 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:43.738306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:43.739549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:43.739549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:43.739549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:43.741971 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:43.789414 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:43.803015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:43.814250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:43.814250 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1277/2538] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:56:44.156514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:44.158036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:44.171955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda771e0-9b82-4b03-a6a8-57602b3eaf54 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:44.176655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:44.181412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:44.181412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:44.181412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] PATH_INFO: `/v3/projects/dcec0de2c3c3477fa85313a1673afa22` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:44.181412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcec0de2c3c3477fa85313a1673afa22)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:44.181412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:44.292205 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:44.300519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcec0de2c3c3477fa85313a1673afa22', 'request_id': 'req-a0586fee-7779-4003-a657-fa49abcb24c0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.305841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcec0de2c3c3477fa85313a1673afa22', 'request_id': 'req-a0586fee-7779-4003-a657-fa49abcb24c0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.312034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcec0de2c3c3477fa85313a1673afa22', 'request_id': 'req-a0586fee-7779-4003-a657-fa49abcb24c0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.449331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] The token cache is being invalidate because project dcec0de2c3c3477fa85313a1673afa22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:44.449647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.450271 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.451834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1262/2539] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:44 2022] DELETE /identity/v3/projects/dcec0de2c3c3477fa85313a1673afa22 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:44.455849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:44.466959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a85bf-ecc6-4660-8946-1695edc64b27 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:44.522946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:44.523741 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:44.524031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:44.524311 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] PATH_INFO: `/v3/users/7c6d8e57a1d341489d23d0a1d1d576d8` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:44.525539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6d8e57a1d341489d23d0a1d1d576d8)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:44.527462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:44.752236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6d8e57a1d341489d23d0a1d1d576d8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.752913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6d8e57a1d341489d23d0a1d1d576d8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.753503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6d8e57a1d341489d23d0a1d1d576d8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.760855 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6d8e57a1d341489d23d0a1d1d576d8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.875142 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3398395e-441d-4d27-8e53-2b95f49c3a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6d8e57a1d341489d23d0a1d1d576d8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:44.893369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1278/2540] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:56:44 2022] DELETE /identity/v3/users/7c6d8e57a1d341489d23d0a1d1d576d8 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:45.267909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:45.269535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:45.283884 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0586fee-7779-4003-a657-fa49abcb24c0 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:45.343779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:45.344750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:45.345113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:45.345453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] PATH_INFO: `/v3/projects/f679f8f994de473bb27a309d706db297` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:45.346699 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f679f8f994de473bb27a309d706db297)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:45.348775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:45.444898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f679f8f994de473bb27a309d706db297', 'request_id': 'req-d522f633-50da-46fd-acbe-d763c6b37b8b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:45.450806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f679f8f994de473bb27a309d706db297', 'request_id': 'req-d522f633-50da-46fd-acbe-d763c6b37b8b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:45.457213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f679f8f994de473bb27a309d706db297', 'request_id': 'req-d522f633-50da-46fd-acbe-d763c6b37b8b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:45.556644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] The token cache is being invalidate because project f679f8f994de473bb27a309d706db297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:56:45.556940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:45.557574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d522f633-50da-46fd-acbe-d763c6b37b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:56:45.559182 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1263/2541] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:56:45 2022] DELETE /identity/v3/projects/f679f8f994de473bb27a309d706db297 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:45.566102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bb8d6-8660-46ec-9e55-493d4967dc7f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:45.566581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bb8d6-8660-46ec-9e55-493d4967dc7f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:45.566690 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bb8d6-8660-46ec-9e55-493d4967dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:45.584782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-903bb8d6-8660-46ec-9e55-493d4967dc7f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:45.589998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-903bb8d6-8660-46ec-9e55-493d4967dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:45.645788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1279/2542] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:56:45.653816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd8e57-6c8f-4606-8934-fbf3ef1a0779 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:45.654121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd8e57-6c8f-4606-8934-fbf3ef1a0779 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:45.654461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd8e57-6c8f-4606-8934-fbf3ef1a0779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:45.672787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b8fd8e57-6c8f-4606-8934-fbf3ef1a0779 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:45.676198 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b8fd8e57-6c8f-4606-8934-fbf3ef1a0779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:45.682890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1264/2543] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:45.687191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bb8d6-8660-46ec-9e55-493d4967dc7f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:45.697899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bb8d6-8660-46ec-9e55-493d4967dc7f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:45.702642 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:45.704718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:45.704718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:45.704718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:45.704718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:56:45.705184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:45.706505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:45.708001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:45.718437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1280/2544] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:56:45.722663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd8e57-6c8f-4606-8934-fbf3ef1a0779 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:45.732391 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd8e57-6c8f-4606-8934-fbf3ef1a0779 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:45.737064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:45.737715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:45.738002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:45.738282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:45.738982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:45.741536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:45.875235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1265/2545] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:45 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:45.878982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:45.889467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd20114-afb6-4fc5-baa9-ab672b4c87de admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:45.945728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:45.946403 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:45.946695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:45.946977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:45.947678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:45.949679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:46.051241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1281/2546] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:45 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:56:46.052545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:46.063650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a63f9-e241-4d87-a029-4f3212c84ed9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:46.068552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:46.069211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:46.069495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:46.069770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:46.070493 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:46.072085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:46.083016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1266/2547] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:46.090000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:46.102178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523416b-bd38-44f1-8781-0371c6c97f48 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:46.107105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb45e7-a963-4f09-a7d9-b804a5406fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:46.107790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb45e7-a963-4f09-a7d9-b804a5406fda admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:46.108080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb45e7-a963-4f09-a7d9-b804a5406fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:46.108476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb45e7-a963-4f09-a7d9-b804a5406fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:46.109130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb45e7-a963-4f09-a7d9-b804a5406fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:46.110935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb45e7-a963-4f09-a7d9-b804a5406fda admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:46.125847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1282/2548] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:46.128318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:46.138897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09611d2-7b54-40fa-ad9a-e26a97c14efb admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:46.143987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:46.145018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:46.145371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:46.145719 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] PATH_INFO: `/v3/projects/be1213650f38494aa2aeb1aadc7ccbc7/users/9c95e9df33ad4f819f26fb8b695df7e9/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:46.158941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:46.162655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1213650f38494aa2aeb1aadc7ccbc7, user_id=9c95e9df33ad4f819f26fb8b695df7e9, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:46.166068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:46.253501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1267/2549] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:46 2022] PUT /identity/v3/projects/be1213650f38494aa2aeb1aadc7ccbc7/users/9c95e9df33ad4f819f26fb8b695df7e9/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:46.260280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b401-fb42-4839-9b46-167add9a0f39 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:46.260605 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b401-fb42-4839-9b46-167add9a0f39 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:46.260915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1b401-fb42-4839-9b46-167add9a0f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:46.280416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ecc1b401-fb42-4839-9b46-167add9a0f39 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:46.281661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ecc1b401-fb42-4839-9b46-167add9a0f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:46.340103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1283/2550] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:46.349805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:46.351744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:46.408863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f772e4-652c-403b-845c-4cf1c478f1c1 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:46.412408 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:46.413473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:46.413756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:46.414035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:46.423007 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:46.424446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:46.434067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1268/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:56:52.028878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757ee39-2a6c-4930-ba9b-b5a629126c8c None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:52.029348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757ee39-2a6c-4930-ba9b-b5a629126c8c None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:52.029841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757ee39-2a6c-4930-ba9b-b5a629126c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:52.070053 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3757ee39-2a6c-4930-ba9b-b5a629126c8c None None] MFA Rules not processed for user `9c95e9df33ad4f819f26fb8b695df7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:52.071373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3757ee39-2a6c-4930-ba9b-b5a629126c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:56:52.148830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1284/2552] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:56:52.153413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:52.163341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0fd45-904f-41d0-a417-ca58cfdbd180 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:52.167950 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:52.168655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:52.168939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:52.169217 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:52.169922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:52.171879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:52.279949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1269/2553] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:56:52 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:56:52.284100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757ee39-2a6c-4930-ba9b-b5a629126c8c None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:52.290442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:56:52.295976 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757ee39-2a6c-4930-ba9b-b5a629126c8c None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:52.352937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:52.353546 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:52.353852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:52.354129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:52.354843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:52.356808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:52.471293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1285/2554] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:56:52 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:56:52.471431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:52.482416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b288fbf-0a21-4074-baea-df2b34e4dfa6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:52.487167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:52.487937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:52.488323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:52.488698 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:52.489501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:52.491880 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:52.503307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1270/2555] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:56:52.507564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:52.517820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31d6ee-48c4-49f0-a05b-89784136fdd6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:52.522651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:52.523548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:52.523836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:52.524126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] PATH_INFO: `/v3/projects/b4291ba018c443d392571201d810b551/users/ef540a5f6a9b4a4980f69d0553e5b058/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:52.544156 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4291ba018c443d392571201d810b551, user_id=ef540a5f6a9b4a4980f69d0553e5b058, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:52.546947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:52.651068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1286/2556] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:52 2022] PUT /identity/v3/projects/b4291ba018c443d392571201d810b551/users/ef540a5f6a9b4a4980f69d0553e5b058/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:52.654990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:52.666486 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51f806-29dd-4eec-b261-cffd24576ecc admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:52.740778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0fbcc-913c-4d60-989e-a91ee8804482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:52.741479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0fbcc-913c-4d60-989e-a91ee8804482 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:52.741769 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0fbcc-913c-4d60-989e-a91ee8804482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:52.742051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0fbcc-913c-4d60-989e-a91ee8804482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:52.742786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0fbcc-913c-4d60-989e-a91ee8804482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:52.744537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0fbcc-913c-4d60-989e-a91ee8804482 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:52.754834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1271/2557] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:56:52.759052 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:56:52.769961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c2f48-2f06-46b3-8f5a-2b6e2ee9b158 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:56:52.775368 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:56:52.776323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:52.776636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:52.776914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] PATH_INFO: `/v3/projects/b4291ba018c443d392571201d810b551/users/ef540a5f6a9b4a4980f69d0553e5b058/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:52.778955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4291ba018c443d392571201d810b551, user_id=ef540a5f6a9b4a4980f69d0553e5b058, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:56:52.781716 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:56:52.794484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1287/2558] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:56:52 2022] PUT /identity/v3/projects/b4291ba018c443d392571201d810b551/users/ef540a5f6a9b4a4980f69d0553e5b058/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:56:59.924847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910962d-0f4b-44bc-a9b1-1073d7ef00c3 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:56:59.925363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910962d-0f4b-44bc-a9b1-1073d7ef00c3 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:56:59.925485 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910962d-0f4b-44bc-a9b1-1073d7ef00c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:56:59.960879 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-2910962d-0f4b-44bc-a9b1-1073d7ef00c3 None None] MFA Rules not processed for user `ef540a5f6a9b4a4980f69d0553e5b058`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:56:59.962516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-2910962d-0f4b-44bc-a9b1-1073d7ef00c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:00.020901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:00.047618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1272/2559] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:56:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:00.060350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:00.062128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:00.149590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5eecc-3650-45e2-aea2-2e71848139d2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:00.155071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:00.155719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:00.156004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:00.156277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:00.204661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:00.212950 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:00.214532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:00.223493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1288/2560] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:00.650831 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910962d-0f4b-44bc-a9b1-1073d7ef00c3 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:00.705437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910962d-0f4b-44bc-a9b1-1073d7ef00c3 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:00.709613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:00.710194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:00.710475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:00.710748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:00.712522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:00.784073 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:00.785670 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:00.796002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1273/2561] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:05.297496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:05.299985 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:05.326766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc716b9-f301-4721-aa86-718703414c68 service nova] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:05.343051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:05.343051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:05.343051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:05.343051 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] PATH_INFO: `/v3/users/311c19d7062648f78ba4c2090c58f14d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:05.343580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=311c19d7062648f78ba4c2090c58f14d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:05.343580 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:05.667227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '311c19d7062648f78ba4c2090c58f14d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:05.667881 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '311c19d7062648f78ba4c2090c58f14d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:05.668516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '311c19d7062648f78ba4c2090c58f14d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:05.675805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '311c19d7062648f78ba4c2090c58f14d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:05.785977 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-448900a4-c25d-46cf-85a6-10a9c7bce2c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '311c19d7062648f78ba4c2090c58f14d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:05.808666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1289/2562] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:57:05 2022] DELETE /identity/v3/users/311c19d7062648f78ba4c2090c58f14d => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:06.076891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:06.089514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fef7b6-7f10-457c-aa5c-f6d135dd8759 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:06.191630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:06.192707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:06.193021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:06.193298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] PATH_INFO: `/v3/projects/1e0c8bf2bfe34311961a4bde7f102944` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:06.194488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0c8bf2bfe34311961a4bde7f102944)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:06.196604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:06.328237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:06.347650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0c8bf2bfe34311961a4bde7f102944', 'request_id': 'req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:06.360263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0c8bf2bfe34311961a4bde7f102944', 'request_id': 'req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:06.367529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0c8bf2bfe34311961a4bde7f102944', 'request_id': 'req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:06.471212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] The token cache is being invalidate because project 1e0c8bf2bfe34311961a4bde7f102944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:57:06.471511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:06.472356 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c9475b5a-e06c-48e9-bc6e-30c3424a75e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:06.475085 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1274/2563] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:57:06 2022] DELETE /identity/v3/projects/1e0c8bf2bfe34311961a4bde7f102944 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:06.480096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb64c-675b-41be-b462-68ef1e125dd6 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:06.480586 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb64c-675b-41be-b462-68ef1e125dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:06.480749 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb64c-675b-41be-b462-68ef1e125dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:06.502267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-661cb64c-675b-41be-b462-68ef1e125dd6 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:06.503681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-661cb64c-675b-41be-b462-68ef1e125dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:06.590110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1290/2564] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:06.597962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6ce33-a32b-46de-9496-0b3f381285b8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:06.598255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6ce33-a32b-46de-9496-0b3f381285b8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:06.598558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6ce33-a32b-46de-9496-0b3f381285b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:06.618298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-c3e6ce33-a32b-46de-9496-0b3f381285b8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:06.619591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c3e6ce33-a32b-46de-9496-0b3f381285b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:06.627707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1275/2565] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:06.632162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cb64c-675b-41be-b462-68ef1e125dd6 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:06.641908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cb64c-675b-41be-b462-68ef1e125dd6 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:06.647122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:06.647782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:06.648068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:06.648484 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:06.649232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:57:06.649544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:06.651180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:06.667086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1291/2566] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:57:06.673173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6ce33-a32b-46de-9496-0b3f381285b8 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:06.682778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6ce33-a32b-46de-9496-0b3f381285b8 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:06.687396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:06.688113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:06.688396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:06.689549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:06.689549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:06.691856 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:06.787295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1276/2567] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:57:06 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:57:06.791453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:06.802907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170cd30-07ca-4afd-b29f-7d82b3b7ebfe admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:06.856808 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:06.866631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:06.867693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:06.867693 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:06.867878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:06.868558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:06.870485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:06.969357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1292/2568] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:57:06 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:57:06.973687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:06.984531 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddae904-20f0-40c9-ba96-5a8092519c52 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:06.989277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:06.989988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:06.990275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:06.990552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:06.991280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:06.992961 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:07.003867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1277/2569] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:57:07.008864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:07.018978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba41e8-07cb-4523-b309-420d5965fed0 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:07.024167 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec9c0f-d022-43b2-ae08-bfa5c3995cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:07.024869 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec9c0f-d022-43b2-ae08-bfa5c3995cad admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:07.025153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec9c0f-d022-43b2-ae08-bfa5c3995cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:07.025429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec9c0f-d022-43b2-ae08-bfa5c3995cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:07.026151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec9c0f-d022-43b2-ae08-bfa5c3995cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:07.027830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec9c0f-d022-43b2-ae08-bfa5c3995cad admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:07.039817 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1293/2570] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:57:07.043658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:07.053724 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776470d-f72b-4bac-aeac-97bfdf795ab3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:07.058353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:07.059225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:07.059513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:07.059791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] PATH_INFO: `/v3/projects/958d3fd7755a4e0b8419e92a26656e2d/users/de1ce4a75440478d957a3a03d41dbffd/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:07.075647 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=958d3fd7755a4e0b8419e92a26656e2d, user_id=de1ce4a75440478d957a3a03d41dbffd, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:07.078375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:07.215377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1278/2571] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:57:07 2022] PUT /identity/v3/projects/958d3fd7755a4e0b8419e92a26656e2d/users/de1ce4a75440478d957a3a03d41dbffd/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:07.220537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bc6c1-7e4b-4a88-b57f-c4574118c595 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:07.220831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bc6c1-7e4b-4a88-b57f-c4574118c595 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:07.221111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bc6c1-7e4b-4a88-b57f-c4574118c595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:07.239532 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-232bc6c1-7e4b-4a88-b57f-c4574118c595 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:07.243941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-232bc6c1-7e4b-4a88-b57f-c4574118c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:07.304013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1294/2572] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:57:07.314377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:07.316320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:07.376853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3ddd6-e070-4178-8099-74b7e0591a75 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:07.380545 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:07.381200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:07.381557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:07.381848 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:07.392140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:07.394150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:07.404847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1279/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:13.484031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcaf9b-b5d0-4ba3-9485-6779c0aa27f5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:13.484625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcaf9b-b5d0-4ba3-9485-6779c0aa27f5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:13.484625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcaf9b-b5d0-4ba3-9485-6779c0aa27f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:13.515901 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1dfcaf9b-b5d0-4ba3-9485-6779c0aa27f5 None None] MFA Rules not processed for user `de1ce4a75440478d957a3a03d41dbffd`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:13.517180 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1dfcaf9b-b5d0-4ba3-9485-6779c0aa27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:13.624342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1295/2574] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:13.639765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:13.641276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:13.708304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:13.725810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5c66-51cb-4d37-84d2-1bb136c0c6c9 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:13.731532 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a14abe-ddfe-4828-b008-492a6822613d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:13.732189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a14abe-ddfe-4828-b008-492a6822613d service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:13.732530 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a14abe-ddfe-4828-b008-492a6822613d service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:13.732824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a14abe-ddfe-4828-b008-492a6822613d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:13.742261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a14abe-ddfe-4828-b008-492a6822613d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:13.744084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a14abe-ddfe-4828-b008-492a6822613d service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:13.754067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1280/2575] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:16.517820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:16.518324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:16.518432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:16.538345 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] MFA Rules not processed for user `de1ce4a75440478d957a3a03d41dbffd`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:16.539571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:16.550607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1296/2576] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:23.877425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=6 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:25.213656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbcbd-9c06-497f-9c1e-556c5cf0097c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:25.217434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbcbd-9c06-497f-9c1e-556c5cf0097c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:25.217434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbcbd-9c06-497f-9c1e-556c5cf0097c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:25.241881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-757cbcbd-9c06-497f-9c1e-556c5cf0097c None None] MFA Rules not processed for user `9c95e9df33ad4f819f26fb8b695df7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:25.243121 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-757cbcbd-9c06-497f-9c1e-556c5cf0097c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:25.316019 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1281/2577] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:26.532519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:35.313738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=37 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:39.523003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:39.524402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:39.601927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625a9e9-bc77-4485-bf58-c56083ff2250 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:39.606338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:39.607197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:39.607482 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:39.607788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] PATH_INFO: `/v3/users/cdd216ed3f474929b7f9c8f8f79df617` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:39.608970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd216ed3f474929b7f9c8f8f79df617)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:39.610871 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:39.926193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd216ed3f474929b7f9c8f8f79df617'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:39.927444 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd216ed3f474929b7f9c8f8f79df617'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:39.928055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd216ed3f474929b7f9c8f8f79df617'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:39.935092 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd216ed3f474929b7f9c8f8f79df617'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:40.051843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd216ed3f474929b7f9c8f8f79df617'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:40.070527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1297/2578] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:57:39 2022] DELETE /identity/v3/users/cdd216ed3f474929b7f9c8f8f79df617 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:40.075135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:40.075595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:40.075715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:40.102503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:40.103707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:40.162162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1282/2579] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:40.171870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:40.173412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:40.245867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:40.265337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bb429-0226-4062-aa97-baebc779eaaf admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:40.269118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:40.269760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:40.270045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:40.270324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:40.279531 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:40.281187 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:40.293432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1298/2580] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:40.910373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:40.912501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:40.924448 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22608b88-3c3b-4204-81dc-6104c64e08e4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:40.929289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:40.930166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:40.930453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:40.930734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] PATH_INFO: `/v3/projects/32674399b00b4253a370f05ad7297215` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:40.931933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32674399b00b4253a370f05ad7297215)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:40.934028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:41.080233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32674399b00b4253a370f05ad7297215', 'request_id': 'req-05cb3816-b635-4bb1-ad76-d25080113e79'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.086130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32674399b00b4253a370f05ad7297215', 'request_id': 'req-05cb3816-b635-4bb1-ad76-d25080113e79'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.093398 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32674399b00b4253a370f05ad7297215', 'request_id': 'req-05cb3816-b635-4bb1-ad76-d25080113e79'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.371224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] The token cache is being invalidate because project 32674399b00b4253a370f05ad7297215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:57:41.371508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.372049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.373628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1283/2581] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:57:40 2022] DELETE /identity/v3/projects/32674399b00b4253a370f05ad7297215 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:41.377669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:41.387941 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3051cb-6f7c-4825-9d7b-d2f624f3ba54 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:41.443795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:41.444742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:41.448418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:41.448418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] PATH_INFO: `/v3/users/bdacaaaa213148d8861ad17a90ca9353` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:41.448418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdacaaaa213148d8861ad17a90ca9353)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:41.448418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:41.677609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdacaaaa213148d8861ad17a90ca9353'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.678296 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdacaaaa213148d8861ad17a90ca9353'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.678909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdacaaaa213148d8861ad17a90ca9353'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.686181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdacaaaa213148d8861ad17a90ca9353'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.771263 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e183ca55-fbb4-4019-bc61-2a2c55ce1c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdacaaaa213148d8861ad17a90ca9353'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:41.788755 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1299/2582] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:57:41 2022] DELETE /identity/v3/users/bdacaaaa213148d8861ad17a90ca9353 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:57:42.235943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:42.237739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:42.253279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb3816-b635-4bb1-ad76-d25080113e79 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:42.310510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:42.311388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:42.311820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:42.312078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] PATH_INFO: `/v3/projects/bb6a8e94e7674bbf81b795fe75be128d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:42.313328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6a8e94e7674bbf81b795fe75be128d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:42.315239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:42.337933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:42.468674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6a8e94e7674bbf81b795fe75be128d', 'request_id': 'req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:42.474002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6a8e94e7674bbf81b795fe75be128d', 'request_id': 'req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:42.480123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6a8e94e7674bbf81b795fe75be128d', 'request_id': 'req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:42.585570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] The token cache is being invalidate because project bb6a8e94e7674bbf81b795fe75be128d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:57:42.585861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:42.586496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8ce59746-2fe9-41e4-8ae2-9b4189efcf29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:42.588039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1284/2583] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:57:42 2022] DELETE /identity/v3/projects/bb6a8e94e7674bbf81b795fe75be128d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:42.593172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf869-d25c-45e1-91f3-9427eaba830a None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:42.594561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf869-d25c-45e1-91f3-9427eaba830a None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:42.594561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf869-d25c-45e1-91f3-9427eaba830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:42.640671 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-97dbf869-d25c-45e1-91f3-9427eaba830a None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:42.641911 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-97dbf869-d25c-45e1-91f3-9427eaba830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:42.670802 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:42.705494 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1300/2584] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:42.713695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe4930-046e-4fb3-a698-a6fa546d2d0c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:42.713991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe4930-046e-4fb3-a698-a6fa546d2d0c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:42.714277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe4930-046e-4fb3-a698-a6fa546d2d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:42.733997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-86fe4930-046e-4fb3-a698-a6fa546d2d0c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:42.735144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-86fe4930-046e-4fb3-a698-a6fa546d2d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:42.743516 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1285/2585] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:42.747764 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbf869-d25c-45e1-91f3-9427eaba830a None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:42.758240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbf869-d25c-45e1-91f3-9427eaba830a None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:42.762892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:42.763562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:42.763849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:42.764137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:42.765037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:57:42.765349 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:42.766963 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:42.779742 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1301/2586] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:57:42.784300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe4930-046e-4fb3-a698-a6fa546d2d0c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:42.794044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe4930-046e-4fb3-a698-a6fa546d2d0c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:42.798718 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:42.799376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:42.799653 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:42.799927 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:42.800721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:42.802634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:42.948707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1286/2587] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:57:42 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:57:42.952978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:42.963335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9369-87eb-4294-b175-420b51b0b857 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.018907 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.019508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.019794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.020080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.021372 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.023291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:43.112343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1302/2588] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:57:42 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:57:43.116657 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:43.126999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507a01d-b748-42d2-81b0-1ba109f17815 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.131665 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.132317 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.132629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.132915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.133643 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.135285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:43.146151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1287/2589] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:57:43.150008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:43.159938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4900ee-5ade-4a18-97e7-39faafe95da8 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.165327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafad6ff-8f4b-4c0a-b26c-2e2cf3bce4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.165988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad6ff-8f4b-4c0a-b26c-2e2cf3bce4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.166273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad6ff-8f4b-4c0a-b26c-2e2cf3bce4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.166551 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad6ff-8f4b-4c0a-b26c-2e2cf3bce4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.167275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafad6ff-8f4b-4c0a-b26c-2e2cf3bce4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.168992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafad6ff-8f4b-4c0a-b26c-2e2cf3bce4d4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:43.178923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1303/2590] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:57:43.182969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:43.193206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e573a6c-a714-40b3-a2e2-0122a36781a9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.197814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.198687 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.198978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.199257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] PATH_INFO: `/v3/projects/b865e9c378ea433da84492bc9dfb53fc/users/4a1b27b5e91d48669ed149b5652dc5ab/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.214061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b865e9c378ea433da84492bc9dfb53fc, user_id=4a1b27b5e91d48669ed149b5652dc5ab, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.216745 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:43.296962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1288/2591] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:57:43 2022] PUT /identity/v3/projects/b865e9c378ea433da84492bc9dfb53fc/users/4a1b27b5e91d48669ed149b5652dc5ab/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:57:43.307080 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074ed8e-4042-4bdc-a49d-6f4546aa57a8 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.307375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074ed8e-4042-4bdc-a49d-6f4546aa57a8 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.307653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074ed8e-4042-4bdc-a49d-6f4546aa57a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.334552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6074ed8e-4042-4bdc-a49d-6f4546aa57a8 None None] MFA Rules not processed for user `4a1b27b5e91d48669ed149b5652dc5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:43.335867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6074ed8e-4042-4bdc-a49d-6f4546aa57a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:43.400803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1304/2592] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:43.405060 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:43.467457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4e83d-645f-42f9-b9e3-25319e1fa8e2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.472142 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.472827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.473118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.473400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.474658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.476634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:43.564807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1289/2593] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:57:43 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:57:43.569519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074ed8e-4042-4bdc-a49d-6f4546aa57a8 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:43.580773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074ed8e-4042-4bdc-a49d-6f4546aa57a8 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.608701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:43.639330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.640028 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.640330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.640662 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.641352 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.643416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:43.866550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1305/2594] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:57:43 2022] POST /identity/v3/users => generated 297 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:57:43.870469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:43.880833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3870d-856c-47f3-b012-61bb9d64c21e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.885785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.886446 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.886728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.887059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.888343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.889488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:43.900663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1290/2595] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:57:43.904982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:43.915411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386e4ce-4733-4968-91b9-2b1c5b47efde admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:43.919970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:43.920897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:43.921211 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:43.921487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] PATH_INFO: `/v3/projects/3b0b0889bd6848f681bbc3277626721a/users/62e88cce19db405e879e724bd81d0404/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:43.936386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0b0889bd6848f681bbc3277626721a, user_id=62e88cce19db405e879e724bd81d0404, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:43.939210 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:44.078215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1306/2596] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:57:43 2022] PUT /identity/v3/projects/3b0b0889bd6848f681bbc3277626721a/users/62e88cce19db405e879e724bd81d0404/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:44.083743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:44.098437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe86065-6981-46b0-a05f-13d95d92ec85 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:44.111930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:44.164363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fec00-0e3c-497b-874c-bfacaca71d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:44.165106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fec00-0e3c-497b-874c-bfacaca71d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:44.165390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fec00-0e3c-497b-874c-bfacaca71d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:44.165668 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fec00-0e3c-497b-874c-bfacaca71d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:44.166726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fec00-0e3c-497b-874c-bfacaca71d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:44.168903 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fec00-0e3c-497b-874c-bfacaca71d16 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:44.181323 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1291/2597] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:57:44.185557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:44.196787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887b507-ed62-4ac4-9740-a961f40bd265 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:44.201539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:44.202429 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:44.202711 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:44.202988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] PATH_INFO: `/v3/projects/3b0b0889bd6848f681bbc3277626721a/users/62e88cce19db405e879e724bd81d0404/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:44.205518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0b0889bd6848f681bbc3277626721a, user_id=62e88cce19db405e879e724bd81d0404, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:44.209579 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:44.221777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1307/2598] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:57:44 2022] PUT /identity/v3/projects/3b0b0889bd6848f681bbc3277626721a/users/62e88cce19db405e879e724bd81d0404/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:57:44.234127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e3ee-64d5-4a64-94b7-f256b8173155 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:44.239092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e3ee-64d5-4a64-94b7-f256b8173155 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:44.239092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e3ee-64d5-4a64-94b7-f256b8173155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:44.262987 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-5562e3ee-64d5-4a64-94b7-f256b8173155 None None] MFA Rules not processed for user `62e88cce19db405e879e724bd81d0404`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:44.264203 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5562e3ee-64d5-4a64-94b7-f256b8173155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:44.334297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1292/2599] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:57:44.346664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:44.349013 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:44.425878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4ad9-a27b-49df-ac3d-32cc87524717 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:44.436042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:44.439547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:44.439547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:44.439547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:44.447740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:44.449216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:44.459560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1308/2600] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:44.674243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e3ee-64d5-4a64-94b7-f256b8173155 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:44.686605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e3ee-64d5-4a64-94b7-f256b8173155 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:44.690429 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:44.691897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:44.691897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:44.691897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:44.752829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:44.754632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:44.763438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1293/2601] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:48.903993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:48.905451 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:48.961183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15100fb3-5e56-4534-95d3-5740afe6bc79 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:48.964795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453755a-64c3-4c81-ba28-1cae9af24314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:48.965434 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453755a-64c3-4c81-ba28-1cae9af24314 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:48.965721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453755a-64c3-4c81-ba28-1cae9af24314 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:48.966018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453755a-64c3-4c81-ba28-1cae9af24314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:48.967812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0453755a-64c3-4c81-ba28-1cae9af24314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:49.007337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:49.030538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453755a-64c3-4c81-ba28-1cae9af24314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:49.031988 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453755a-64c3-4c81-ba28-1cae9af24314 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:49.042020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1309/2602] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:54.820824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=83 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:58.368783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:58.432394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e978a-a213-4fec-b1d2-c84533e12a6f service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:58.437150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:58.438050 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:58.438340 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:58.438622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] PATH_INFO: `/v3/users/4a1b27b5e91d48669ed149b5652dc5ab` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:58.439866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a1b27b5e91d48669ed149b5652dc5ab)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:58.441861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:58.652079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a1b27b5e91d48669ed149b5652dc5ab'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:58.664256 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a1b27b5e91d48669ed149b5652dc5ab'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:58.665053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a1b27b5e91d48669ed149b5652dc5ab'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:58.665704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a1b27b5e91d48669ed149b5652dc5ab'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:58.672572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a1b27b5e91d48669ed149b5652dc5ab'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:58.780678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1294/2603] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:57:58 2022] DELETE /identity/v3/users/4a1b27b5e91d48669ed149b5652dc5ab => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:58.785591 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:58.786757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:58.786757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:58.804481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:57:58.805723 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:58.866082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1310/2604] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:57:58.876326 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:58.877841 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:58.936357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0961501-a0ce-422b-a155-eb541042331b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:58.940020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:58.940715 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:58.941001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:58.941280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:58.950623 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:58.952084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:58.962011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1295/2605] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:57:59.447109 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:59.448608 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:57:59.461493 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083473e-89f6-4d38-a9ae-40600dc35dcb None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:59.466358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:59.467228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:59.467539 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:59.467914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] PATH_INFO: `/v3/projects/b865e9c378ea433da84492bc9dfb53fc` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:59.469163 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b865e9c378ea433da84492bc9dfb53fc)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:59.471140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:57:59.620792 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b865e9c378ea433da84492bc9dfb53fc', 'request_id': 'req-ef5da66e-3937-443a-8fc5-416629019baa'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:59.627099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b865e9c378ea433da84492bc9dfb53fc', 'request_id': 'req-ef5da66e-3937-443a-8fc5-416629019baa'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:59.723079 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b865e9c378ea433da84492bc9dfb53fc', 'request_id': 'req-ef5da66e-3937-443a-8fc5-416629019baa'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:59.734297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] The token cache is being invalidate because project b865e9c378ea433da84492bc9dfb53fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:57:59.741919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:59.741919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:57:59.741919 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1311/2606] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:57:59 2022] DELETE /identity/v3/projects/b865e9c378ea433da84492bc9dfb53fc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:57:59.742245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:57:59.751137 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea2a20-c2cb-441c-ba61-b3745e06d274 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:57:59.769320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:57:59.807606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:57:59.808538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:57:59.808819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:57:59.809101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] PATH_INFO: `/v3/users/62e88cce19db405e879e724bd81d0404` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:57:59.810288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e88cce19db405e879e724bd81d0404)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:57:59.812204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:00.097707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e88cce19db405e879e724bd81d0404'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:00.111771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e88cce19db405e879e724bd81d0404'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:00.112515 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e88cce19db405e879e724bd81d0404'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:00.113135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e88cce19db405e879e724bd81d0404'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:00.120721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8217b8c6-7cda-4cbd-9e6d-77cb73a465ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e88cce19db405e879e724bd81d0404'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:00.353461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1296/2607] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:57:59 2022] DELETE /identity/v3/users/62e88cce19db405e879e724bd81d0404 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:00.910041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:00.911534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:00.925147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da66e-3937-443a-8fc5-416629019baa admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:00.976523 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:00.981997 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:00.982856 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:00.983137 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:00.983413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] PATH_INFO: `/v3/projects/3b0b0889bd6848f681bbc3277626721a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:00.984655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0b0889bd6848f681bbc3277626721a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:00.986547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:01.120757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0b0889bd6848f681bbc3277626721a', 'request_id': 'req-50e838ec-b385-4f6d-9c99-f10c112cf945'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:01.126860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0b0889bd6848f681bbc3277626721a', 'request_id': 'req-50e838ec-b385-4f6d-9c99-f10c112cf945'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:01.245501 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0b0889bd6848f681bbc3277626721a', 'request_id': 'req-50e838ec-b385-4f6d-9c99-f10c112cf945'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:01.256884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] The token cache is being invalidate because project 3b0b0889bd6848f681bbc3277626721a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:01.257175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:01.257831 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-50e838ec-b385-4f6d-9c99-f10c112cf945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:01.259415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1312/2608] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:00 2022] DELETE /identity/v3/projects/3b0b0889bd6848f681bbc3277626721a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:01.264306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089bdc2-62d3-41b8-b468-03496fec5b4d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.264808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089bdc2-62d3-41b8-b468-03496fec5b4d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.264944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089bdc2-62d3-41b8-b468-03496fec5b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.282742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8089bdc2-62d3-41b8-b468-03496fec5b4d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:01.283936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8089bdc2-62d3-41b8-b468-03496fec5b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:01.341477 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1297/2609] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:01.349462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450da690-91b7-49b7-becd-9a05b63bb6b9 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.349759 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450da690-91b7-49b7-becd-9a05b63bb6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.350046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450da690-91b7-49b7-becd-9a05b63bb6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.369258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-450da690-91b7-49b7-becd-9a05b63bb6b9 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:01.370550 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-450da690-91b7-49b7-becd-9a05b63bb6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:01.379518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1313/2610] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:01.384389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089bdc2-62d3-41b8-b468-03496fec5b4d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:01.395329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089bdc2-62d3-41b8-b468-03496fec5b4d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:01.403003 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:01.403984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.404273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.404605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.405354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:58:01.405896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:01.407303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:01.421655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1298/2611] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 17:58:01.423841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450da690-91b7-49b7-becd-9a05b63bb6b9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:01.433753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450da690-91b7-49b7-becd-9a05b63bb6b9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:01.438527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:01.439183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.439472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.439754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.440509 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:01.443027 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:01.541836 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1314/2612] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:01 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:01.546153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:01.556294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35de2c-dec3-4b14-9334-5eb8730a02b9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:01.583717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:01.613396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:01.614065 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.614349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.614628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.615327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:01.617298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:01.731404 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1299/2613] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:01 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:01.735447 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:01.746363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199432a4-bea6-4e4e-aeec-c445d2a8e815 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:01.750960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:01.751665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.751954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.752236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.753004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:01.754596 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:01.764947 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1315/2614] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:01.768886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:01.778702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe1610-2bdd-4279-aa00-53338360a94d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:01.783334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbfd59-6579-479f-bb2f-0893a3bfbc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:01.783998 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbfd59-6579-479f-bb2f-0893a3bfbc7b admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.784281 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbfd59-6579-479f-bb2f-0893a3bfbc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.784595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbfd59-6579-479f-bb2f-0893a3bfbc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.785321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbfd59-6579-479f-bb2f-0893a3bfbc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:01.787469 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbfd59-6579-479f-bb2f-0893a3bfbc7b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:01.798192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1300/2615] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:01.802620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:01.812654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1fe13-9487-4f50-b0a1-c1222fe09b80 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:01.817284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:01.818161 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.818707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.818987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] PATH_INFO: `/v3/projects/4af31443d33a4fceb3bc8537c1d5d41b/users/379473f9f5bb420d9be920ef980be279/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:01.834014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af31443d33a4fceb3bc8537c1d5d41b, user_id=379473f9f5bb420d9be920ef980be279, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:01.836674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:01.970961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1316/2616] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:01 2022] PUT /identity/v3/projects/4af31443d33a4fceb3bc8537c1d5d41b/users/379473f9f5bb420d9be920ef980be279/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:01.982622 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf9ca-12f6-41d0-9e83-678f7002d4f4 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:01.982914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf9ca-12f6-41d0-9e83-678f7002d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:01.983507 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf9ca-12f6-41d0-9e83-678f7002d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.010255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-22ebf9ca-12f6-41d0-9e83-678f7002d4f4 None None] MFA Rules not processed for user `379473f9f5bb420d9be920ef980be279`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:02.011511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-22ebf9ca-12f6-41d0-9e83-678f7002d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:02.074514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1301/2617] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:02.078886 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:02.142340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658bc06-580a-4c00-b05f-25f834428184 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:02.147629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:02.148289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.148611 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.148891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.149597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:02.151506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:02.278270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1317/2618] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:02 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:02.282349 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf9ca-12f6-41d0-9e83-678f7002d4f4 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:02.293834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf9ca-12f6-41d0-9e83-678f7002d4f4 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:02.350302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:02.350902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.351190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.351492 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.352209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:02.354164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:02.463829 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1302/2619] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:02 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:02.467921 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:02.478659 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b761f7b-87e7-47f4-b3d7-f663437781f2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:02.483396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:02.484082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.484416 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.484661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.485381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:02.486957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:02.489030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:02.497670 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1318/2620] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:02.501774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:02.512061 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8733b7-55c1-434d-96f1-b2bc70e50e11 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:02.516882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:02.517748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.518032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.518312 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] PATH_INFO: `/v3/projects/84b2b1e1beb14c958662cb7a35edd131/users/361d990f1b6e4802bac8d5688592cf0a/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.533133 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b2b1e1beb14c958662cb7a35edd131, user_id=361d990f1b6e4802bac8d5688592cf0a, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:02.535810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:02.638080 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1303/2621] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:02 2022] PUT /identity/v3/projects/84b2b1e1beb14c958662cb7a35edd131/users/361d990f1b6e4802bac8d5688592cf0a/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:02.641973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:02.651909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20227a60-891e-40cd-8242-ba14793ff57e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:02.714131 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5ec1f-2c71-40f0-b495-00c32e2f74ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:02.714813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ec1f-2c71-40f0-b495-00c32e2f74ca admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.715099 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ec1f-2c71-40f0-b495-00c32e2f74ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.715377 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ec1f-2c71-40f0-b495-00c32e2f74ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.716113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5ec1f-2c71-40f0-b495-00c32e2f74ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:02.717761 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5ec1f-2c71-40f0-b495-00c32e2f74ca admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:02.728762 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1319/2622] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:02.732992 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:02.742387 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:02.748806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da864ac-95d4-46a5-a7ff-5242e9a8ebb2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:02.753500 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:02.756977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.756977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.756977 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] PATH_INFO: `/v3/projects/84b2b1e1beb14c958662cb7a35edd131/users/361d990f1b6e4802bac8d5688592cf0a/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.757462 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b2b1e1beb14c958662cb7a35edd131, user_id=361d990f1b6e4802bac8d5688592cf0a, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:02.760113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:02.770144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1304/2623] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:02 2022] PUT /identity/v3/projects/84b2b1e1beb14c958662cb7a35edd131/users/361d990f1b6e4802bac8d5688592cf0a/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:02.780461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5385f-200c-452b-ac8d-785d22b3e110 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.780771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5385f-200c-452b-ac8d-785d22b3e110 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.781048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5385f-200c-452b-ac8d-785d22b3e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.807316 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b8e5385f-200c-452b-ac8d-785d22b3e110 None None] MFA Rules not processed for user `361d990f1b6e4802bac8d5688592cf0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:02.808563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b8e5385f-200c-452b-ac8d-785d22b3e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:02.879031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1320/2624] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:02.894586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:02.894586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:02.969247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8ce99-893a-429a-8a2d-922f711cddd7 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:02.972817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d0bb2-6983-4a17-bea6-06211b259118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:02.973969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0bb2-6983-4a17-bea6-06211b259118 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:02.974248 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0bb2-6983-4a17-bea6-06211b259118 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:02.974522 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d0bb2-6983-4a17-bea6-06211b259118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:02.983431 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d0bb2-6983-4a17-bea6-06211b259118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:02.984890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d0bb2-6983-4a17-bea6-06211b259118 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:02.994693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1305/2625] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:03.026728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5385f-200c-452b-ac8d-785d22b3e110 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:03.088549 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5385f-200c-452b-ac8d-785d22b3e110 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:03.093012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:03.093889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:03.094184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:03.094460 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] PATH_INFO: `/v3/users/379473f9f5bb420d9be920ef980be279` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:03.095681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=379473f9f5bb420d9be920ef980be279)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:03.097748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:03.120869 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:03.345442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379473f9f5bb420d9be920ef980be279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:03.346106 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379473f9f5bb420d9be920ef980be279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:03.346713 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379473f9f5bb420d9be920ef980be279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:03.357441 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379473f9f5bb420d9be920ef980be279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:03.448758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379473f9f5bb420d9be920ef980be279'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:03.467786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1321/2626] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:03 2022] DELETE /identity/v3/users/379473f9f5bb420d9be920ef980be279 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:03.472279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:03.472779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:03.472900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:03.490728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:03.491933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:03.549098 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1306/2627] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:03.559587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:03.561304 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:03.618450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8891a7-41d5-431c-9fe5-2de885e9f276 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:03.621975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:03.622626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:03.622918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:03.623200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:03.632719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:03.634155 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:03.643504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1322/2628] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:04.157649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:04.159191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:04.170979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9527e74-ac1b-4507-9333-bd1e98c5d119 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:04.175810 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:04.177363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:04.177639 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:04.177969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] PATH_INFO: `/v3/projects/4af31443d33a4fceb3bc8537c1d5d41b` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:04.179135 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af31443d33a4fceb3bc8537c1d5d41b)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:04.181124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:04.306497 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af31443d33a4fceb3bc8537c1d5d41b', 'request_id': 'req-676d978e-6c4d-4577-885e-5ea2a6de8645'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.312126 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af31443d33a4fceb3bc8537c1d5d41b', 'request_id': 'req-676d978e-6c4d-4577-885e-5ea2a6de8645'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.319244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af31443d33a4fceb3bc8537c1d5d41b', 'request_id': 'req-676d978e-6c4d-4577-885e-5ea2a6de8645'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.323733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:04.406487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] The token cache is being invalidate because project 4af31443d33a4fceb3bc8537c1d5d41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:04.406681 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.407296 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.408953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1307/2629] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:04 2022] DELETE /identity/v3/projects/4af31443d33a4fceb3bc8537c1d5d41b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:58:04.413714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:04.423388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7ce53-cc4c-4b7e-9c6b-2dc6be8f0a57 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:04.482246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:04.483121 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:04.483409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:04.483687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] PATH_INFO: `/v3/users/361d990f1b6e4802bac8d5688592cf0a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:04.484908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361d990f1b6e4802bac8d5688592cf0a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:04.487126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:04.672383 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:04.675617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361d990f1b6e4802bac8d5688592cf0a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.676385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361d990f1b6e4802bac8d5688592cf0a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.677562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361d990f1b6e4802bac8d5688592cf0a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.684485 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361d990f1b6e4802bac8d5688592cf0a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.820357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a3304d52-02bc-444c-85c0-1880b0095999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361d990f1b6e4802bac8d5688592cf0a'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:04.839350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1323/2630] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:04 2022] DELETE /identity/v3/users/361d990f1b6e4802bac8d5688592cf0a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:05.296565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:05.298238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:05.311277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d978e-6c4d-4577-885e-5ea2a6de8645 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:05.369096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:05.369988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:05.370300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:05.370553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] PATH_INFO: `/v3/projects/84b2b1e1beb14c958662cb7a35edd131` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:05.371866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b2b1e1beb14c958662cb7a35edd131)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:05.373832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:05.483931 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b2b1e1beb14c958662cb7a35edd131', 'request_id': 'req-c5d31644-bf67-40b3-905c-2770840e314b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:05.489937 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b2b1e1beb14c958662cb7a35edd131', 'request_id': 'req-c5d31644-bf67-40b3-905c-2770840e314b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:05.496048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b2b1e1beb14c958662cb7a35edd131', 'request_id': 'req-c5d31644-bf67-40b3-905c-2770840e314b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:05.579763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] The token cache is being invalidate because project 84b2b1e1beb14c958662cb7a35edd131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:05.579981 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:05.580628 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c5d31644-bf67-40b3-905c-2770840e314b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:05.582174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1308/2631] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:05 2022] DELETE /identity/v3/projects/84b2b1e1beb14c958662cb7a35edd131 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:05.591661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396411-44a1-4f23-b73f-ddc031f4e6b1 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:05.592171 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396411-44a1-4f23-b73f-ddc031f4e6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:05.592272 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396411-44a1-4f23-b73f-ddc031f4e6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:05.612049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-f6396411-44a1-4f23-b73f-ddc031f4e6b1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:05.613288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f6396411-44a1-4f23-b73f-ddc031f4e6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:05.679512 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1324/2632] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:05.686967 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03f2d-2358-42e5-99da-1cd3b8694f22 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:05.687267 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03f2d-2358-42e5-99da-1cd3b8694f22 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:05.687593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03f2d-2358-42e5-99da-1cd3b8694f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:05.706922 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d0c03f2d-2358-42e5-99da-1cd3b8694f22 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:05.708166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d0c03f2d-2358-42e5-99da-1cd3b8694f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:05.720655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1309/2633] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:05.725358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396411-44a1-4f23-b73f-ddc031f4e6b1 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:05.737606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396411-44a1-4f23-b73f-ddc031f4e6b1 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:05.742965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:05.743630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:05.743933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:05.744218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:05.745035 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:58:05.745343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:05.747286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:05.761324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1325/2634] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:58:05.765857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03f2d-2358-42e5-99da-1cd3b8694f22 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:05.775329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03f2d-2358-42e5-99da-1cd3b8694f22 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:05.780177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:05.783418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:05.783418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:05.783418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:05.783418 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:05.783975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:05.902458 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1310/2635] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:05 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:05.906557 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:05.918873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d392d2-7326-415a-aa05-d60cc156665a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:05.977320 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:05.977993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:05.978280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:05.978559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:05.979259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:05.981221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.091717 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1326/2636] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:05 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:06.096162 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:06.108236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac98e09-98da-4661-b0ff-51c2e674dcd9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:06.113109 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:06.113784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.114070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.114373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.115191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:06.116837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.127707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1311/2637] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:58:06.132365 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:06.142559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec05fa-16ff-4f89-a1b5-931f565a9bfd admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:06.147896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864b1e8-3cdc-4540-9d59-74bfbd5c53da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:06.148603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864b1e8-3cdc-4540-9d59-74bfbd5c53da admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.148896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864b1e8-3cdc-4540-9d59-74bfbd5c53da admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.149179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864b1e8-3cdc-4540-9d59-74bfbd5c53da admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.149915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864b1e8-3cdc-4540-9d59-74bfbd5c53da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:06.151521 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864b1e8-3cdc-4540-9d59-74bfbd5c53da admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.167861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1327/2638] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 17:58:06.172105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:06.182420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b9804-d866-4002-a273-7ac04c13fc04 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:06.186932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:06.187808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.188096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.188369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] PATH_INFO: `/v3/projects/d42aca09e6f6437b99eb461cea6e4e59/users/0b503f4349d645a190d21212f4f7020a/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.205567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42aca09e6f6437b99eb461cea6e4e59, user_id=0b503f4349d645a190d21212f4f7020a, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:06.210257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.320102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1312/2639] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:06 2022] PUT /identity/v3/projects/d42aca09e6f6437b99eb461cea6e4e59/users/0b503f4349d645a190d21212f4f7020a/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:06.331291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91969d-6b91-4560-a5cd-2d63534bdb4f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.331783 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91969d-6b91-4560-a5cd-2d63534bdb4f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.332009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91969d-6b91-4560-a5cd-2d63534bdb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.363464 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ec91969d-6b91-4560-a5cd-2d63534bdb4f None None] MFA Rules not processed for user `0b503f4349d645a190d21212f4f7020a`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:06.364924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ec91969d-6b91-4560-a5cd-2d63534bdb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:06.377691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:06.432286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1328/2640] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:06.436852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:06.462262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:06.498105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28019d05-d50c-4739-887f-a9884201754e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:06.502940 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:06.503601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.503887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.504165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.505475 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:06.507376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.633818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1313/2641] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:06 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:06.638953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91969d-6b91-4560-a5cd-2d63534bdb4f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:06.656005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91969d-6b91-4560-a5cd-2d63534bdb4f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:06.715918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:06.716415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.716731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.717004 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.717695 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:06.719998 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.843852 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1329/2642] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:06 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:06.848588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:06.859425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3ce32-54b3-4b74-8e8b-b9e3a0c7210b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:06.864077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:06.865322 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.865605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.865877 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.866596 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:06.868191 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.878534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1314/2643] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:06.884396 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:06.894686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ce0d2-6ad4-4e7d-bce2-9bef56601ae7 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:06.899361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:06.900346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:06.900667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:06.900946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] PATH_INFO: `/v3/projects/78329066db5c4af78a05f16695a16008/users/c5e552f04a874be3955b6652c807d920/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:06.916273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78329066db5c4af78a05f16695a16008, user_id=c5e552f04a874be3955b6652c807d920, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:06.919058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:06.995014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1330/2644] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:06 2022] PUT /identity/v3/projects/78329066db5c4af78a05f16695a16008/users/c5e552f04a874be3955b6652c807d920/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:06.999187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:07.009339 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870638a-c062-40bf-80c7-fc9cff1bac18 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:07.064337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff83f3-4d68-441a-b065-cef49a6e5b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:07.065032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff83f3-4d68-441a-b065-cef49a6e5b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:07.065318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff83f3-4d68-441a-b065-cef49a6e5b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:07.065597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff83f3-4d68-441a-b065-cef49a6e5b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:07.066324 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff83f3-4d68-441a-b065-cef49a6e5b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:07.067955 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff83f3-4d68-441a-b065-cef49a6e5b8f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:07.077852 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1315/2645] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:07.082001 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:07.092872 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262af2e-0526-448e-bf15-32476598fbc4 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:07.097678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:07.098632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:07.098923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:07.099207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] PATH_INFO: `/v3/projects/78329066db5c4af78a05f16695a16008/users/c5e552f04a874be3955b6652c807d920/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:07.101294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=78329066db5c4af78a05f16695a16008, user_id=c5e552f04a874be3955b6652c807d920, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:07.104008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:07.115924 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1331/2646] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:07 2022] PUT /identity/v3/projects/78329066db5c4af78a05f16695a16008/users/c5e552f04a874be3955b6652c807d920/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:07.126142 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28250b-7513-4f7b-9c06-fbb503116c22 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:07.126427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28250b-7513-4f7b-9c06-fbb503116c22 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:07.126702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28250b-7513-4f7b-9c06-fbb503116c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:07.154016 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-ec28250b-7513-4f7b-9c06-fbb503116c22 None None] MFA Rules not processed for user `c5e552f04a874be3955b6652c807d920`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:07.155255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ec28250b-7513-4f7b-9c06-fbb503116c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:07.220911 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1316/2647] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:58:07.231865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:07.233627 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:07.291992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0ac0d-bc46-4d5f-a90a-7bbdcd43fe2b admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:07.295572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089036a2-4afb-4f27-96ac-f341d1865fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:07.296216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089036a2-4afb-4f27-96ac-f341d1865fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:07.296543 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089036a2-4afb-4f27-96ac-f341d1865fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:07.296820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089036a2-4afb-4f27-96ac-f341d1865fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:07.306242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089036a2-4afb-4f27-96ac-f341d1865fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:07.307707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089036a2-4afb-4f27-96ac-f341d1865fb1 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:07.317694 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1332/2648] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:08.566078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28250b-7513-4f7b-9c06-fbb503116c22 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:08.586509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:08.630189 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28250b-7513-4f7b-9c06-fbb503116c22 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:08.634743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:08.635605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:08.635888 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:08.636163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] PATH_INFO: `/v3/users/0b503f4349d645a190d21212f4f7020a` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:08.637576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b503f4349d645a190d21212f4f7020a)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:08.639569 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:08.882310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b503f4349d645a190d21212f4f7020a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:08.893840 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b503f4349d645a190d21212f4f7020a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:08.894514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b503f4349d645a190d21212f4f7020a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:08.895193 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b503f4349d645a190d21212f4f7020a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:08.901963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b503f4349d645a190d21212f4f7020a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:08.999517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1317/2649] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:08 2022] DELETE /identity/v3/users/0b503f4349d645a190d21212f4f7020a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:09.004118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:09.005164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:09.005265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:09.023050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:09.024264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:09.029260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:09.081520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1333/2650] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:58:09.091269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:09.092744 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:09.152765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eae40-702d-4ac8-bd65-15c39b312c1e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:09.156211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:09.156926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:09.157244 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:09.157543 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:09.167126 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:09.168678 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:09.178732 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1318/2651] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:09.593012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:09.594615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:09.608337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0354238-212e-4e30-bece-5aed45f71611 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:09.612992 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:09.615753 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:09.616049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:09.616337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] PATH_INFO: `/v3/projects/d42aca09e6f6437b99eb461cea6e4e59` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:09.620794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42aca09e6f6437b99eb461cea6e4e59)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:09.620794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:09.804192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42aca09e6f6437b99eb461cea6e4e59', 'request_id': 'req-0bca4d77-a4da-45cf-9940-e5d74fc0912e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:09.810628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42aca09e6f6437b99eb461cea6e4e59', 'request_id': 'req-0bca4d77-a4da-45cf-9940-e5d74fc0912e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:09.897970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42aca09e6f6437b99eb461cea6e4e59', 'request_id': 'req-0bca4d77-a4da-45cf-9940-e5d74fc0912e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:09.909766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] The token cache is being invalidate because project d42aca09e6f6437b99eb461cea6e4e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:09.912415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:09.912415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:09.915339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1334/2652] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:09 2022] DELETE /identity/v3/projects/d42aca09e6f6437b99eb461cea6e4e59 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:09.916736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:09.931163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f257a8-d9a5-4c4d-bed3-bd086513fe83 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:09.978037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:09.996461 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:09.997327 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:09.997618 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:09.997900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] PATH_INFO: `/v3/users/c5e552f04a874be3955b6652c807d920` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:09.999090 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e552f04a874be3955b6652c807d920)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:10.001091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:10.195789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e552f04a874be3955b6652c807d920'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:10.208891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e552f04a874be3955b6652c807d920'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:10.214378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e552f04a874be3955b6652c807d920'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:10.214378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e552f04a874be3955b6652c807d920'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:10.217625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-76b6b69a-12df-4e54-9f72-5ed0e751f4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e552f04a874be3955b6652c807d920'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:10.296501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1319/2653] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:09 2022] DELETE /identity/v3/users/c5e552f04a874be3955b6652c807d920 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:10.748293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:10.749798 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:10.765335 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4d77-a4da-45cf-9940-e5d74fc0912e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:10.828769 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:10.831337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:10.831337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:10.831337 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] PATH_INFO: `/v3/projects/78329066db5c4af78a05f16695a16008` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:10.831603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] RBAC: Authorizing `identity:delete_project(project_id=78329066db5c4af78a05f16695a16008)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:10.833958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:10.964402 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:10.974708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78329066db5c4af78a05f16695a16008', 'request_id': 'req-7cbd33f0-2332-445e-96d7-d7752641177f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:10.981676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78329066db5c4af78a05f16695a16008', 'request_id': 'req-7cbd33f0-2332-445e-96d7-d7752641177f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:11.079513 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78329066db5c4af78a05f16695a16008', 'request_id': 'req-7cbd33f0-2332-445e-96d7-d7752641177f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:11.091175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] The token cache is being invalidate because project 78329066db5c4af78a05f16695a16008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:11.091470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:11.092283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-7cbd33f0-2332-445e-96d7-d7752641177f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:11.093922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1335/2654] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:10 2022] DELETE /identity/v3/projects/78329066db5c4af78a05f16695a16008 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:11.099123 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba33a2-7d1e-4c31-b23e-7a75a69ff35b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.099626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba33a2-7d1e-4c31-b23e-7a75a69ff35b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.099765 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba33a2-7d1e-4c31-b23e-7a75a69ff35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.118835 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-54ba33a2-7d1e-4c31-b23e-7a75a69ff35b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:11.120056 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-54ba33a2-7d1e-4c31-b23e-7a75a69ff35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:11.179291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1320/2655] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:11.187088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7aff5-4721-4e24-9b26-8168e922930e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.187682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7aff5-4721-4e24-9b26-8168e922930e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.187682 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7aff5-4721-4e24-9b26-8168e922930e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.205813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-2ed7aff5-4721-4e24-9b26-8168e922930e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:11.207014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2ed7aff5-4721-4e24-9b26-8168e922930e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:11.215518 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1336/2656] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:11.219646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba33a2-7d1e-4c31-b23e-7a75a69ff35b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:11.230100 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba33a2-7d1e-4c31-b23e-7a75a69ff35b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:11.234748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:11.241830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.241830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.241830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.241830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:58:11.241830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:11.241830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:11.252901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1321/2657] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:58:11.257587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7aff5-4721-4e24-9b26-8168e922930e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:11.266734 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7aff5-4721-4e24-9b26-8168e922930e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:11.271953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:11.272653 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.272945 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.273229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.273915 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:11.275805 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:11.427554 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1337/2658] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:11 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:11.431641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:11.442357 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cac75-112f-47f6-9b85-bd608a01db21 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:11.486959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:11.499434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:11.500103 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.500386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.500691 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.501405 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:11.503321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:11.715009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1322/2659] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:11 2022] POST /identity/v3/users => generated 292 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:11.719975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:11.730902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368df8f-d32a-4594-afc3-1d4db64c5df7 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:11.735541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:11.736209 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.736542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.736812 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.737615 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:11.739193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:11.751042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1338/2660] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:11.754952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:11.765277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79b865-9ec3-4e8a-88a3-941b86f26c71 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:11.771527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c29adf-d7e3-43d9-a88b-47cbd98d7d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:11.772716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c29adf-d7e3-43d9-a88b-47cbd98d7d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.773249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c29adf-d7e3-43d9-a88b-47cbd98d7d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.773774 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c29adf-d7e3-43d9-a88b-47cbd98d7d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.775046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c29adf-d7e3-43d9-a88b-47cbd98d7d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:11.777164 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c29adf-d7e3-43d9-a88b-47cbd98d7d9a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:11.789072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1323/2661] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:11.793530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:11.803701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9776f-a6fc-4c26-bc40-0c5fe69f0bba admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:11.808215 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:11.809144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:11.809428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:11.809708 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] PATH_INFO: `/v3/projects/2c4cec0b2431483694c4c9715e8a672b/users/624aca1d978747b7a7bffa5950df83a0/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:11.824757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4cec0b2431483694c4c9715e8a672b, user_id=624aca1d978747b7a7bffa5950df83a0, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:11.827718 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:11.965790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1339/2662] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:11 2022] PUT /identity/v3/projects/2c4cec0b2431483694c4c9715e8a672b/users/624aca1d978747b7a7bffa5950df83a0/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:58:12.030672 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4378f4-cc20-41c9-9629-5eeb9e058777 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:12.030894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4378f4-cc20-41c9-9629-5eeb9e058777 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:12.031163 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4378f4-cc20-41c9-9629-5eeb9e058777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:12.056658 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bc4378f4-cc20-41c9-9629-5eeb9e058777 None None] MFA Rules not processed for user `624aca1d978747b7a7bffa5950df83a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:12.057876 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bc4378f4-cc20-41c9-9629-5eeb9e058777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:12.131140 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1324/2663] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:12.135851 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:12.203340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f584d7-02f6-4506-81fd-17eb387748d6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:12.209203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:12.209865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:12.210262 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:12.210666 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:12.211520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:12.213813 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:12.367649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1340/2664] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:12 2022] POST /identity/v3/projects => generated 381 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:12.372094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4378f4-cc20-41c9-9629-5eeb9e058777 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:12.382770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4378f4-cc20-41c9-9629-5eeb9e058777 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:12.443012 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:12.443933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:12.444213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:12.444561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:12.445409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:12.447726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:12.657155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1325/2665] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:12 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:12.661062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:12.671252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a5a10-8e4e-4a96-bf7e-dd61a67977a9 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:12.675796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:12.676978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:12.677268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:12.677555 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:12.678280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:12.679917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:12.690584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1341/2666] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:12.695610 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:12.705842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fea511-f55d-490b-b0b6-c3612578292d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:12.711203 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:12.712102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:12.712388 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:12.712700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] PATH_INFO: `/v3/projects/815cc5d5a4884db3af4929a7e5831974/users/1edeeb0a17a54602a9c72b72c41c95f3/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:12.729158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=815cc5d5a4884db3af4929a7e5831974, user_id=1edeeb0a17a54602a9c72b72c41c95f3, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:12.731894 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:12.891376 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1326/2667] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:12 2022] PUT /identity/v3/projects/815cc5d5a4884db3af4929a7e5831974/users/1edeeb0a17a54602a9c72b72c41c95f3/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:12.895980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:12.905910 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aff7db-a7b4-409d-a989-77bcec7fc3f5 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:12.937008 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:12.961259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6a7b7-90f1-4a8d-a547-cab81faa757b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:12.961916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6a7b7-90f1-4a8d-a547-cab81faa757b admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:12.962201 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6a7b7-90f1-4a8d-a547-cab81faa757b admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:12.962476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6a7b7-90f1-4a8d-a547-cab81faa757b admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:12.963188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6a7b7-90f1-4a8d-a547-cab81faa757b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:12.964840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6a7b7-90f1-4a8d-a547-cab81faa757b admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:12.974716 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1342/2668] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:12.978722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:12.989710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992b1e1-54cb-4fc3-9c27-c20200be3ca3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:12.994558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:12.995451 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:12.995846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:12.996009 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] PATH_INFO: `/v3/projects/815cc5d5a4884db3af4929a7e5831974/users/1edeeb0a17a54602a9c72b72c41c95f3/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:12.998083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=815cc5d5a4884db3af4929a7e5831974, user_id=1edeeb0a17a54602a9c72b72c41c95f3, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:13.000770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:13.013450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1327/2669] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:12 2022] PUT /identity/v3/projects/815cc5d5a4884db3af4929a7e5831974/users/1edeeb0a17a54602a9c72b72c41c95f3/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:13.024445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442317-ad99-4166-bb10-5d8ca28dc3c2 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:13.024757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442317-ad99-4166-bb10-5d8ca28dc3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:13.025043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442317-ad99-4166-bb10-5d8ca28dc3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:13.050905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e4442317-ad99-4166-bb10-5d8ca28dc3c2 None None] MFA Rules not processed for user `1edeeb0a17a54602a9c72b72c41c95f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:13.052145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e4442317-ad99-4166-bb10-5d8ca28dc3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:13.123415 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1343/2670] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:13.135650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:13.137393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:13.147853 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:13.195907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661982a3-da8e-46b2-9573-d8f1610ec6b5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:13.199905 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:13.200630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:13.200882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:13.201158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:13.211442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:13.212915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:13.226425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1328/2671] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:13.371530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4442317-ad99-4166-bb10-5d8ca28dc3c2 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:13.381848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4442317-ad99-4166-bb10-5d8ca28dc3c2 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:13.385453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e1c23-e264-4b32-b397-0bf43fef6432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:13.386093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e1c23-e264-4b32-b397-0bf43fef6432 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:13.386380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e1c23-e264-4b32-b397-0bf43fef6432 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:13.386715 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e1c23-e264-4b32-b397-0bf43fef6432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:13.443202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e1c23-e264-4b32-b397-0bf43fef6432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:13.445446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e1c23-e264-4b32-b397-0bf43fef6432 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:13.454933 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1344/2672] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:14.049363 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:14.115386 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023bd9-9076-4276-9f71-f2eae2a1d38e service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:14.120113 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:14.121069 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:14.121334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:14.121615 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] PATH_INFO: `/v3/users/624aca1d978747b7a7bffa5950df83a0` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:14.122795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624aca1d978747b7a7bffa5950df83a0)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:14.124730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:14.304255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624aca1d978747b7a7bffa5950df83a0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:14.316110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624aca1d978747b7a7bffa5950df83a0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:14.317334 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624aca1d978747b7a7bffa5950df83a0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:14.323581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624aca1d978747b7a7bffa5950df83a0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:14.324978 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624aca1d978747b7a7bffa5950df83a0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:14.417148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1329/2673] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:14 2022] DELETE /identity/v3/users/624aca1d978747b7a7bffa5950df83a0 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:14.421469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:14.421946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:14.422049 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:14.439448 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:14.440697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:14.489144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:14.500158 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1345/2674] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:14.510425 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:14.511913 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:14.545614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:14.571147 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df1902-8ea0-4ef4-a931-ad8910fa3a08 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:14.574663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:14.580298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:14.580298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:14.580298 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:14.584890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:14.586330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:14.595805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1330/2675] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:15.134172 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:15.135639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:15.153774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4153372-4c40-48de-993d-17fe24ed0e7e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:15.167600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:15.168936 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:15.169287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:15.169752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] PATH_INFO: `/v3/projects/2c4cec0b2431483694c4c9715e8a672b` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:15.171130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4cec0b2431483694c4c9715e8a672b)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:15.173242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:15.294940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4cec0b2431483694c4c9715e8a672b', 'request_id': 'req-469648d9-af0a-4a89-a2c5-91e719feca88'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.301196 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4cec0b2431483694c4c9715e8a672b', 'request_id': 'req-469648d9-af0a-4a89-a2c5-91e719feca88'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.368757 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4cec0b2431483694c4c9715e8a672b', 'request_id': 'req-469648d9-af0a-4a89-a2c5-91e719feca88'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.378956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] The token cache is being invalidate because project 2c4cec0b2431483694c4c9715e8a672b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:15.379280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.380426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.382023 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1346/2676] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:15 2022] DELETE /identity/v3/projects/2c4cec0b2431483694c4c9715e8a672b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:15.386059 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:15.396264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce059-b2e5-4ea2-9eb3-c872497fe1cd service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:15.453450 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:15.454306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:15.454588 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:15.454861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] PATH_INFO: `/v3/users/1edeeb0a17a54602a9c72b72c41c95f3` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:15.456067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1edeeb0a17a54602a9c72b72c41c95f3)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:15.458535 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:15.677036 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1edeeb0a17a54602a9c72b72c41c95f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.689192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1edeeb0a17a54602a9c72b72c41c95f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.689898 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1edeeb0a17a54602a9c72b72c41c95f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.690503 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1edeeb0a17a54602a9c72b72c41c95f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.698134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-475b35af-9800-44df-8864-acbd9731050d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1edeeb0a17a54602a9c72b72c41c95f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:15.782195 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1331/2677] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:15 2022] DELETE /identity/v3/users/1edeeb0a17a54602a9c72b72c41c95f3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:16.243162 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:16.244703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:16.257594 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469648d9-af0a-4a89-a2c5-91e719feca88 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:16.324105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:16.325057 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:16.325344 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:16.325626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] PATH_INFO: `/v3/projects/815cc5d5a4884db3af4929a7e5831974` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:16.326841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815cc5d5a4884db3af4929a7e5831974)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:16.328816 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:16.478358 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815cc5d5a4884db3af4929a7e5831974', 'request_id': 'req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:16.484540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815cc5d5a4884db3af4929a7e5831974', 'request_id': 'req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:16.662587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815cc5d5a4884db3af4929a7e5831974', 'request_id': 'req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:16.673665 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] The token cache is being invalidate because project 815cc5d5a4884db3af4929a7e5831974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:16.673952 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:16.674655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8ab6a2c3-83c2-4cbf-ab45-9a4e5db15806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:16.676388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1347/2678] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:16 2022] DELETE /identity/v3/projects/815cc5d5a4884db3af4929a7e5831974 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:16.681320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805e23e-30fc-466b-9b95-5392333af60f None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:16.681800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805e23e-30fc-466b-9b95-5392333af60f None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:16.681900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805e23e-30fc-466b-9b95-5392333af60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:16.701722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8805e23e-30fc-466b-9b95-5392333af60f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:16.702166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8805e23e-30fc-466b-9b95-5392333af60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:16.748509 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:16.763038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1332/2679] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:16.772598 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af86be-c9f7-4630-aec3-c18a442523a4 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:16.772927 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af86be-c9f7-4630-aec3-c18a442523a4 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:16.773166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af86be-c9f7-4630-aec3-c18a442523a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:16.794084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c1af86be-c9f7-4630-aec3-c18a442523a4 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:16.795354 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c1af86be-c9f7-4630-aec3-c18a442523a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:16.804276 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1348/2680] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:16.808526 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805e23e-30fc-466b-9b95-5392333af60f None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:16.818866 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805e23e-30fc-466b-9b95-5392333af60f None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:16.823447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:16.824091 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:16.824371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:16.824683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:16.825406 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:58:16.825712 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:16.827291 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:16.839487 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1333/2681] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:58:16.844310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af86be-c9f7-4630-aec3-c18a442523a4 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:16.848846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:16.854423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af86be-c9f7-4630-aec3-c18a442523a4 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:16.859075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:16.859728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:16.860041 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:16.860319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:16.861045 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:16.862961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:17.008063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1349/2682] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:16 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:17.012329 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:17.023132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dbca9-6e3f-48f4-98b4-e63b5ddcb869 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:17.079318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:17.079988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:17.080275 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:17.080599 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:17.081303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:17.083266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:17.200310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1334/2683] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:17 2022] POST /identity/v3/users => generated 281 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:17.204499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:17.214578 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8288d8-a677-4d77-9dff-d130adeeb568 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:17.219534 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:17.220202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:17.220571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:17.228519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:17.228519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:17.228519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:17.234407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1350/2684] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:17.238367 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:17.249247 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05aa2-5cb9-4a6a-841b-54a922b9b450 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:17.253918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677dbb1b-196b-45e5-8e1e-be860fd5e7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:17.254583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dbb1b-196b-45e5-8e1e-be860fd5e7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:17.254944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dbb1b-196b-45e5-8e1e-be860fd5e7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:17.255220 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dbb1b-196b-45e5-8e1e-be860fd5e7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:17.255951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677dbb1b-196b-45e5-8e1e-be860fd5e7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:17.257708 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677dbb1b-196b-45e5-8e1e-be860fd5e7b5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:17.272572 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1335/2685] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:17.274427 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:17.284643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f07e1-e0e7-4619-8393-f3fc33d6a872 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:17.289869 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:17.290758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:17.291048 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:17.291332 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] PATH_INFO: `/v3/projects/f697ff35e7884ccc91cf508572a54c95/users/a9f7705f43784d819a97282a2acf75b1/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:17.306626 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f697ff35e7884ccc91cf508572a54c95, user_id=a9f7705f43784d819a97282a2acf75b1, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:17.309314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:17.420393 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1351/2686] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:17 2022] PUT /identity/v3/projects/f697ff35e7884ccc91cf508572a54c95/users/a9f7705f43784d819a97282a2acf75b1/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:17.431095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6475-e7b4-4cc2-9c3c-3a29c4711af5 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:17.431389 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6475-e7b4-4cc2-9c3c-3a29c4711af5 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:17.431675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6475-e7b4-4cc2-9c3c-3a29c4711af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:17.458629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-882f6475-e7b4-4cc2-9c3c-3a29c4711af5 None None] MFA Rules not processed for user `a9f7705f43784d819a97282a2acf75b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:17.459945 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-882f6475-e7b4-4cc2-9c3c-3a29c4711af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:17.524594 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1336/2687] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:17.556544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:17.558267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:17.614865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2757-8de7-4b99-b84e-796f8f439b51 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:17.618520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa82e64-8067-4ffa-bc9e-3158a0094861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:17.619189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa82e64-8067-4ffa-bc9e-3158a0094861 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:17.619473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa82e64-8067-4ffa-bc9e-3158a0094861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:17.619751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa82e64-8067-4ffa-bc9e-3158a0094861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:17.627838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa82e64-8067-4ffa-bc9e-3158a0094861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:17.629301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa82e64-8067-4ffa-bc9e-3158a0094861 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:17.638214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1352/2688] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:17.652484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6475-e7b4-4cc2-9c3c-3a29c4711af5 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:17.700746 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:17.715541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6475-e7b4-4cc2-9c3c-3a29c4711af5 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:17.720239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:17.721157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:17.721444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:17.721726 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] PATH_INFO: `/v3/users/a9f7705f43784d819a97282a2acf75b1` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:17.722936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f7705f43784d819a97282a2acf75b1)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:17.724915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:17.926663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f7705f43784d819a97282a2acf75b1'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:17.938463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f7705f43784d819a97282a2acf75b1'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:17.939156 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f7705f43784d819a97282a2acf75b1'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:17.939785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f7705f43784d819a97282a2acf75b1'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:17.946887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f7705f43784d819a97282a2acf75b1'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:18.022773 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1337/2689] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:17 2022] DELETE /identity/v3/users/a9f7705f43784d819a97282a2acf75b1 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:18.027306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:18.027597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:18.027906 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:18.044787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:18.045987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:18.106236 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1353/2690] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 17:58:18.115822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:18.117254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:18.173320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287de5c-5aec-42eb-854b-10ce16673158 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:18.177072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eee71e-a4ec-4ff9-a824-153de3eac675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:18.177659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eee71e-a4ec-4ff9-a824-153de3eac675 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:18.177944 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eee71e-a4ec-4ff9-a824-153de3eac675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:18.178221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eee71e-a4ec-4ff9-a824-153de3eac675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:18.187626 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eee71e-a4ec-4ff9-a824-153de3eac675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:18.189105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eee71e-a4ec-4ff9-a824-153de3eac675 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:18.199400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1338/2691] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:18.707407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:18.707407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:18.719340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9debc4-73f1-4d72-8a47-75d27849965e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:18.724898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:18.725697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:18.725951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:18.726227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] PATH_INFO: `/v3/projects/f697ff35e7884ccc91cf508572a54c95` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:18.727432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f697ff35e7884ccc91cf508572a54c95)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:18.729338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:18.915033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:18.918600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f697ff35e7884ccc91cf508572a54c95', 'request_id': 'req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:18.924674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f697ff35e7884ccc91cf508572a54c95', 'request_id': 'req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:19.065993 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f697ff35e7884ccc91cf508572a54c95', 'request_id': 'req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:19.077417 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] The token cache is being invalidate because project f697ff35e7884ccc91cf508572a54c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:19.077702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:19.078323 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94fbb9fa-7772-4fb9-a6ff-798b7e7d4646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:19.079850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1354/2692] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:18 2022] DELETE /identity/v3/projects/f697ff35e7884ccc91cf508572a54c95 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:19.085280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d79ca-7527-4089-82b0-acde510c9e40 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.085736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d79ca-7527-4089-82b0-acde510c9e40 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.085846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d79ca-7527-4089-82b0-acde510c9e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.108279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-938d79ca-7527-4089-82b0-acde510c9e40 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:19.108279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-938d79ca-7527-4089-82b0-acde510c9e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:19.165105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1339/2693] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:19.173303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0fb2c-ff2a-46f2-b008-462ff8b79a84 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.173602 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0fb2c-ff2a-46f2-b008-462ff8b79a84 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.173888 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0fb2c-ff2a-46f2-b008-462ff8b79a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.192496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c7b0fb2c-ff2a-46f2-b008-462ff8b79a84 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:19.193942 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c7b0fb2c-ff2a-46f2-b008-462ff8b79a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:19.201904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1355/2694] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:19.206224 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d79ca-7527-4089-82b0-acde510c9e40 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:19.216364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d79ca-7527-4089-82b0-acde510c9e40 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:19.220957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:19.221612 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.221902 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.222184 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.222918 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:58:19.223791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:19.225427 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:19.227761 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:19.237743 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1340/2695] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:58:19.242699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0fb2c-ff2a-46f2-b008-462ff8b79a84 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:19.252005 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0fb2c-ff2a-46f2-b008-462ff8b79a84 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:19.256640 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:19.257282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.257563 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.257837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.258514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:19.260400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:19.384126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1356/2696] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:19 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:19.388249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:19.398928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530ea96-8e1f-470d-b879-0e374846bfc9 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:19.455890 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:19.456589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.456915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.457174 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.458348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:19.460246 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:19.553528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1341/2697] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:19 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:19.557737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:19.568935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d23b2-c117-48e6-bcf5-243aa3370b1f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:19.573744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:19.574386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.574673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.574956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.575722 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:19.577351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:19.588326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1357/2698] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:19.592297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:19.602379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081db8ee-d8db-44ce-b1b2-2abe9e00aaad admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:19.606989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea699c-94c7-4009-851a-f89e815bd838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:19.607648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea699c-94c7-4009-851a-f89e815bd838 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.607933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea699c-94c7-4009-851a-f89e815bd838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.608212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea699c-94c7-4009-851a-f89e815bd838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.608979 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea699c-94c7-4009-851a-f89e815bd838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:19.610576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea699c-94c7-4009-851a-f89e815bd838 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:19.620558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1342/2699] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:19.624633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:19.635058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bc6cc-36e5-4044-ba86-4a697a7e7b01 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:19.639630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:19.640529 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.640814 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.641093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] PATH_INFO: `/v3/projects/4e8785ea25e241fdabde4d52c8fb225c/users/e1a7ea89517b457b80e4640d1399f2d7/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.655896 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8785ea25e241fdabde4d52c8fb225c, user_id=e1a7ea89517b457b80e4640d1399f2d7, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:19.658620 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:19.783075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1358/2700] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:19 2022] PUT /identity/v3/projects/4e8785ea25e241fdabde4d52c8fb225c/users/e1a7ea89517b457b80e4640d1399f2d7/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:19.794120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c94869-2833-467b-9f6b-12eb4d32ad1a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.794412 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c94869-2833-467b-9f6b-12eb4d32ad1a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.794695 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c94869-2833-467b-9f6b-12eb4d32ad1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.825006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-14c94869-2833-467b-9f6b-12eb4d32ad1a None None] MFA Rules not processed for user `e1a7ea89517b457b80e4640d1399f2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:19.826243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-14c94869-2833-467b-9f6b-12eb4d32ad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:19.899817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1343/2701] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:19.907638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:19.970754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768cbde-a3ce-4ea2-8832-8dd5f8febd7a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:19.976059 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:19.976784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:19.977064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:19.983842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:19.983842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:19.983842 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:20.109889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1359/2702] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:19 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:20.113889 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c94869-2833-467b-9f6b-12eb4d32ad1a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:20.124714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c94869-2833-467b-9f6b-12eb4d32ad1a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:20.185037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:20.185710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:20.185989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:20.186270 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:20.186984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:20.188915 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:20.197863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:20.325983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1344/2703] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:20 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 17:58:20.330153 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:20.340867 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36143b85-75d2-4fef-936e-02044dfe009d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:20.345430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:20.346086 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:20.346375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:20.346655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:20.347381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:20.349064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:20.359504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1360/2704] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:20.363559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:20.373491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e22ff-4607-4139-bf2d-ccd00a82e82f admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:20.378083 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:20.379004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:20.382314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:20.382608 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] PATH_INFO: `/v3/projects/6b09dd954ea348978e7bb7848d8a5cc3/users/9eb3c9fffd6a412bbf7e9faabfec8f0d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:20.399817 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b09dd954ea348978e7bb7848d8a5cc3, user_id=9eb3c9fffd6a412bbf7e9faabfec8f0d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:20.402674 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:20.782933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1345/2705] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:20 2022] PUT /identity/v3/projects/6b09dd954ea348978e7bb7848d8a5cc3/users/9eb3c9fffd6a412bbf7e9faabfec8f0d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:20.787020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:20.796975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091889b-ca11-4fbe-b264-a09c677948cf admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:20.841778 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:20.857200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ac4b-d7de-463f-a567-383496165e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:20.857880 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ac4b-d7de-463f-a567-383496165e66 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:20.858169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ac4b-d7de-463f-a567-383496165e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:20.858452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ac4b-d7de-463f-a567-383496165e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:20.859176 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ac4b-d7de-463f-a567-383496165e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:20.860825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ac4b-d7de-463f-a567-383496165e66 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:20.870891 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1361/2706] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:20.875506 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:20.886316 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac99a63-bbfd-4ba2-95d9-b7f80474bcf3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:20.890887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:20.891770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:20.892128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:20.892457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] PATH_INFO: `/v3/projects/6b09dd954ea348978e7bb7848d8a5cc3/users/9eb3c9fffd6a412bbf7e9faabfec8f0d/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:20.894440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b09dd954ea348978e7bb7848d8a5cc3, user_id=9eb3c9fffd6a412bbf7e9faabfec8f0d, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:20.897754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:20.908632 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1346/2707] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:20 2022] PUT /identity/v3/projects/6b09dd954ea348978e7bb7848d8a5cc3/users/9eb3c9fffd6a412bbf7e9faabfec8f0d/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:58:20.919130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5fda-995b-4608-b60d-c36561d39806 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:20.919502 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5fda-995b-4608-b60d-c36561d39806 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:20.919857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c5fda-995b-4608-b60d-c36561d39806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:20.948946 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-164c5fda-995b-4608-b60d-c36561d39806 None None] MFA Rules not processed for user `9eb3c9fffd6a412bbf7e9faabfec8f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:20.950264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-164c5fda-995b-4608-b60d-c36561d39806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:21.017828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1362/2708] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:21.029447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:21.031142 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:21.090442 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0b944-a15f-4517-8bf8-52378f556392 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:21.094072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6201ef4-aa71-4275-b8c0-9d5c222ea58a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:21.094713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6201ef4-aa71-4275-b8c0-9d5c222ea58a service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:21.094996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6201ef4-aa71-4275-b8c0-9d5c222ea58a service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:21.095273 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6201ef4-aa71-4275-b8c0-9d5c222ea58a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:21.153032 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6201ef4-aa71-4275-b8c0-9d5c222ea58a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:21.154490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6201ef4-aa71-4275-b8c0-9d5c222ea58a service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:21.163716 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1347/2709] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:24.460016 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c5fda-995b-4608-b60d-c36561d39806 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:24.535617 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c5fda-995b-4608-b60d-c36561d39806 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:24.541124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:24.542034 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:24.542321 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:24.542603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] PATH_INFO: `/v3/users/e1a7ea89517b457b80e4640d1399f2d7` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:24.543865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a7ea89517b457b80e4640d1399f2d7)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:24.545964 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:24.800413 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a7ea89517b457b80e4640d1399f2d7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:24.801029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a7ea89517b457b80e4640d1399f2d7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:24.801629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a7ea89517b457b80e4640d1399f2d7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:24.808581 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a7ea89517b457b80e4640d1399f2d7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:24.879837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a7ea89517b457b80e4640d1399f2d7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:24.882169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:24.898929 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1363/2710] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:24 2022] DELETE /identity/v3/users/e1a7ea89517b457b80e4640d1399f2d7 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 17:58:24.903676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42482d88-7306-4579-8521-7821f030b5ec None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:24.904166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42482d88-7306-4579-8521-7821f030b5ec None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:24.904333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42482d88-7306-4579-8521-7821f030b5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:24.921310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-42482d88-7306-4579-8521-7821f030b5ec None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:24.922529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-42482d88-7306-4579-8521-7821f030b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:24.991359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1348/2711] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:25.001190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:25.002639 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:25.060777 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449eb098-1540-4df0-8c89-03884f6e1ead admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:25.064589 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:25.065240 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:25.065530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:25.065833 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:25.076177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:25.077655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:25.088033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1364/2712] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:25.595449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42482d88-7306-4579-8521-7821f030b5ec None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:25.596957 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-42482d88-7306-4579-8521-7821f030b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:25.609964 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42482d88-7306-4579-8521-7821f030b5ec None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:25.615630 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:25.616533 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:25.616821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:25.617099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] PATH_INFO: `/v3/projects/4e8785ea25e241fdabde4d52c8fb225c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:25.618297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8785ea25e241fdabde4d52c8fb225c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:25.620221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:25.628864 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:25.735685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8785ea25e241fdabde4d52c8fb225c', 'request_id': 'req-1c08de06-900c-4d55-bee9-eb45990a1261'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:25.741803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8785ea25e241fdabde4d52c8fb225c', 'request_id': 'req-1c08de06-900c-4d55-bee9-eb45990a1261'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:25.748965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8785ea25e241fdabde4d52c8fb225c', 'request_id': 'req-1c08de06-900c-4d55-bee9-eb45990a1261'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:25.844857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] The token cache is being invalidate because project 4e8785ea25e241fdabde4d52c8fb225c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:25.845148 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:25.845808 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:25.847473 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1349/2713] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:25 2022] DELETE /identity/v3/projects/4e8785ea25e241fdabde4d52c8fb225c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:25.852132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:25.861902 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5996e0-6a1a-4534-8f69-a73daf69bac3 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:25.917239 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:25.918115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:25.918382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:25.918657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] PATH_INFO: `/v3/users/9eb3c9fffd6a412bbf7e9faabfec8f0d` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:25.919819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb3c9fffd6a412bbf7e9faabfec8f0d)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:25.921799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:26.304253 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb3c9fffd6a412bbf7e9faabfec8f0d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:26.304935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb3c9fffd6a412bbf7e9faabfec8f0d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:26.305542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb3c9fffd6a412bbf7e9faabfec8f0d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:26.312188 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb3c9fffd6a412bbf7e9faabfec8f0d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:26.408610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0012c7e8-eabe-4657-949b-68a0da25d09d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb3c9fffd6a412bbf7e9faabfec8f0d'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:26.426235 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1365/2714] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:25 2022] DELETE /identity/v3/users/9eb3c9fffd6a412bbf7e9faabfec8f0d => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:26.962985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:26.964645 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:26.982299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08de06-900c-4d55-bee9-eb45990a1261 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:27.047263 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:27.048250 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:27.048567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:27.048846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] PATH_INFO: `/v3/projects/6b09dd954ea348978e7bb7848d8a5cc3` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:27.050053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b09dd954ea348978e7bb7848d8a5cc3)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:27.052553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:27.174908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b09dd954ea348978e7bb7848d8a5cc3', 'request_id': 'req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:27.180784 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b09dd954ea348978e7bb7848d8a5cc3', 'request_id': 'req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:27.186965 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b09dd954ea348978e7bb7848d8a5cc3', 'request_id': 'req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:27.279338 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] The token cache is being invalidate because project 6b09dd954ea348978e7bb7848d8a5cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:27.279637 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:27.280280 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-129dddbe-2639-4b49-93a1-a4ed81f0b3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:27.281845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1350/2715] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:26 2022] DELETE /identity/v3/projects/6b09dd954ea348978e7bb7848d8a5cc3 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:27.287649 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575a5a8-05da-4aa2-b66b-be802948a5ab None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:27.288135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575a5a8-05da-4aa2-b66b-be802948a5ab None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:27.288255 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575a5a8-05da-4aa2-b66b-be802948a5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:27.296126 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:27.307541 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-6575a5a8-05da-4aa2-b66b-be802948a5ab None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:27.308847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6575a5a8-05da-4aa2-b66b-be802948a5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:27.372288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1366/2716] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:27.380197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cf4c4-1524-4009-917c-7b1edcaa9287 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:27.380546 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cf4c4-1524-4009-917c-7b1edcaa9287 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:27.380861 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cf4c4-1524-4009-917c-7b1edcaa9287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:27.399800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-0e7cf4c4-1524-4009-917c-7b1edcaa9287 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:27.401218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0e7cf4c4-1524-4009-917c-7b1edcaa9287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:27.409204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1351/2717] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:27.414044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575a5a8-05da-4aa2-b66b-be802948a5ab None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:27.423719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575a5a8-05da-4aa2-b66b-be802948a5ab None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:27.428847 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:27.429510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:27.429799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:27.430078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:27.430804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 17:58:27.431112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:27.432765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:27.444828 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1367/2718] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 17:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 17:58:27.449039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cf4c4-1524-4009-917c-7b1edcaa9287 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:27.458986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cf4c4-1524-4009-917c-7b1edcaa9287 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:27.463722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:27.464396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:27.464730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:27.465771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:27.465771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:27.467795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:27.471956 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:27.474317 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:27.483494 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:27.564022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8de7-8b26-4859-b6e1-35776a0dcf02 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:27.569488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:27.570134 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:27.570424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:27.570709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:27.572675 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:27.584411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:27.586095 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:27.595897 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1368/2719] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 17:58:27.610402 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1352/2720] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 17:58:27 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:27.614310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:27.624326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281aeec-c268-4ece-a355-a064b69ef338 service placement] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:27.697015 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:27.697614 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:27.697900 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:27.698181 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:27.699273 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:27.701636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:27.740362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:27.991796 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1369/2721] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 17:58:27 2022] POST /identity/v3/users => generated 287 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 17:58:27.993633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:28.006371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6041231-53d2-4a6b-937b-d065a31add89 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:28.012909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:28.013671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:28.014110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:28.015075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:28.015954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:28.017789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:28.032304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1353/2722] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:28.035983 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:28.046118 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1add5-73a2-4bec-8b22-31ac6f5b79fd admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:28.050768 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8443c0-e008-4521-a705-cdd24ee01027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:28.051785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8443c0-e008-4521-a705-cdd24ee01027 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:28.052143 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8443c0-e008-4521-a705-cdd24ee01027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:28.052530 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8443c0-e008-4521-a705-cdd24ee01027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:28.053510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8443c0-e008-4521-a705-cdd24ee01027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:28.055232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8443c0-e008-4521-a705-cdd24ee01027 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:28.070363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1370/2723] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 17:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 17:58:28.071685 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:28.084150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ac0f-e0d8-4eb5-900b-1d16ec9eab39 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:28.088488 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:28.089561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:28.089925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:28.090262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] PATH_INFO: `/v3/projects/e7446c8b8b0c432180c2612f71e764cb/users/2d377a4733fd4621a85fcc8c1b2203da/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:28.111186 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7446c8b8b0c432180c2612f71e764cb, user_id=2d377a4733fd4621a85fcc8c1b2203da, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:28.114463 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:28.260943 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1354/2724] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 17:58:28 2022] PUT /identity/v3/projects/e7446c8b8b0c432180c2612f71e764cb/users/2d377a4733fd4621a85fcc8c1b2203da/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:28.272032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:28.272329 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:28.272647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:28.300204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] MFA Rules not processed for user `2d377a4733fd4621a85fcc8c1b2203da`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:28.301938 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:28.365789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1371/2725] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:28.376375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:28.378264 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:28.443368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bde1f-1706-4743-8166-58cff9a871cd admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:28.447315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:28.447960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:28.448371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:28.448806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:28.456949 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:28.458916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:28.468145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1355/2726] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:29.832895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:29.834348 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:29.891506 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d22c49-d438-4aff-8e9a-890e7a96b13e None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:29.896208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:29.896899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:29.897218 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:29.897499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:29.899361 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:29.909223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:29.911214 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:29.919405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1372/2727] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:30.031636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:30.042124 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83144fc6-da36-44c3-87ca-558b64eacff8 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:30.045857 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c794b8d-08fd-422b-8817-d10340d7ef0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:30.046527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794b8d-08fd-422b-8817-d10340d7ef0d service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:30.046798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794b8d-08fd-422b-8817-d10340d7ef0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:30.047079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794b8d-08fd-422b-8817-d10340d7ef0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:30.048972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-6c794b8d-08fd-422b-8817-d10340d7ef0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:30.144930 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794b8d-08fd-422b-8817-d10340d7ef0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:30.146421 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794b8d-08fd-422b-8817-d10340d7ef0d service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:30.168689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1356/2728] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:39.915735 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:INSERT=2,keystone:SELECT=65 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:40.166024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:40.496022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:40.563370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f80d0a-a900-4162-b7c6-fae2f89219c0 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:40.568625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:40.569837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:40.570644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:40.571050 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] PATH_INFO: `/v3/users/2d377a4733fd4621a85fcc8c1b2203da` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:40.572293 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d377a4733fd4621a85fcc8c1b2203da)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:40.574397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:40.812944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d377a4733fd4621a85fcc8c1b2203da'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:40.813610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d377a4733fd4621a85fcc8c1b2203da'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:40.814212 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d377a4733fd4621a85fcc8c1b2203da'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:40.821882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d377a4733fd4621a85fcc8c1b2203da'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:40.918676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d377a4733fd4621a85fcc8c1b2203da'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:40.936937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1373/2729] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 17:58:40 2022] DELETE /identity/v3/users/2d377a4733fd4621a85fcc8c1b2203da => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:40.941952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:40.942255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:40.942536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:40.960262 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:58:40.963623 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:41.020821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1357/2730] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:58:41.030453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:41.031922 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:41.096918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfb5c6-fce8-4c23-9006-2b71479a2c57 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:41.101194 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:41.101846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:41.102130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:41.102408 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:41.118070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:41.120920 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:41.131663 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1374/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:58:41.681434 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:58:41.683084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:58:41.697218 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0126d9-8115-4197-9f05-fefc3c25b22b None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:58:41.702381 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:58:41.703750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:58:41.703750 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:58:41.703986 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] PATH_INFO: `/v3/projects/e7446c8b8b0c432180c2612f71e764cb` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:58:41.705223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7446c8b8b0c432180c2612f71e764cb)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:58:41.707187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:58:41.860210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7446c8b8b0c432180c2612f71e764cb', 'request_id': 'req-31df183a-428a-4198-8fe6-6d51544a27df'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:41.866227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7446c8b8b0c432180c2612f71e764cb', 'request_id': 'req-31df183a-428a-4198-8fe6-6d51544a27df'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:41.872232 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7446c8b8b0c432180c2612f71e764cb', 'request_id': 'req-31df183a-428a-4198-8fe6-6d51544a27df'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:41.992011 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] The token cache is being invalidate because project e7446c8b8b0c432180c2612f71e764cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 17:58:41.992242 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:41.992832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 17:58:41.994371 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1358/2732] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 17:58:41 2022] DELETE /identity/v3/projects/e7446c8b8b0c432180c2612f71e764cb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 17:58:51.127151 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:58:51.988725 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:59:24.759362 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:59:24.760830 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:24.771850 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07827ed6-0092-4e87-b614-bf8069c5de4b service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:59:24.819203 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2749d2-547e-4501-84d6-f9c7c78cfcd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:59:24.820102 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2749d2-547e-4501-84d6-f9c7c78cfcd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:59:24.820389 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2749d2-547e-4501-84d6-f9c7c78cfcd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:59:24.820714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2749d2-547e-4501-84d6-f9c7c78cfcd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:59:24.822472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6c2749d2-547e-4501-84d6-f9c7c78cfcd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:24.889667 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2749d2-547e-4501-84d6-f9c7c78cfcd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:59:24.891091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2749d2-547e-4501-84d6-f9c7c78cfcd6 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:59:24.902046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1375/2733] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:59:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:59:28.399235 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:59:28.400693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:28.412885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df183a-428a-4198-8fe6-6d51544a27df admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:59:28.416565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:59:28.417167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:59:28.417447 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:59:28.417721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:59:28.419483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:28.433378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:59:28.434803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:59:28.446342 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1359/2734] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:59:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:59:34.897164 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=54 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:59:38.441201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=6 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 17:59:54.396174 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:59:54.396712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:59:54.396819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:59:54.413560 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] MFA Rules not processed for user `9c95e9df33ad4f819f26fb8b695df7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 17:59:54.414785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:54.470249 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1376/2735] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 17:59:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 17:59:57.032556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:59:57.043793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:57.048319 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a34c2-335e-4240-85a0-5a11902cb779 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:59:57.054520 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:59:57.055230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:59:57.055517 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:59:57.055847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:59:57.057733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:57.073337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:59:57.075157 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:59:57.091640 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1360/2736] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:59:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 17:59:57.178306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 17:59:57.179955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:57.193136 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d8187-8f82-4fe4-a2b1-529ed0391eff None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 17:59:57.197058 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 17:59:57.197702 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 17:59:57.197986 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 17:59:57.198259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 17:59:57.200038 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 17:59:57.212773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 17:59:57.214712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 17:59:57.226192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1377/2737] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 17:59:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:00:07.083577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=6 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:00:07.219144 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=31 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:00:08.675995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:08.677656 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:08.766416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20a594-37db-4a3d-a1a8-87c69512b910 service nova] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:08.771651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:08.772304 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:08.772650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:08.772928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:08.774748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:08.789683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:08.789683 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:08.800907 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1361/2738] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:00:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:00:10.461248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:10.470327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30bcb4-9228-47d8-bc5a-51bfbbd797d0 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:10.474754 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:10.475514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:10.475876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:10.476234 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:10.478264 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:10.546775 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:10.548208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:10.559496 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1378/2739] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:00:14.407337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:14.408997 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:14.429294 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a6ab4-599f-4e46-bab2-cb02c08ce8a7 service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:14.434985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:14.436115 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:14.436415 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:14.438728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] PATH_INFO: `/v3/users/0fcc8275481e41f38bbdfac7db67b2e5` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:14.438728 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcc8275481e41f38bbdfac7db67b2e5)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:14.444202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:14.726249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcc8275481e41f38bbdfac7db67b2e5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:14.738284 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcc8275481e41f38bbdfac7db67b2e5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:14.739026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcc8275481e41f38bbdfac7db67b2e5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:14.739654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcc8275481e41f38bbdfac7db67b2e5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:14.748255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcc8275481e41f38bbdfac7db67b2e5'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:14.872037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1362/2740] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:00:14 2022] DELETE /identity/v3/users/0fcc8275481e41f38bbdfac7db67b2e5 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:00:15.229455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:15.245468 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466106-a269-4c8a-b328-d6f5f57e57d6 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:15.311905 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:15.312895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:15.313184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:15.313467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] PATH_INFO: `/v3/projects/863e2e49ffb74a889fb88c4c2e7edcd4` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:15.314696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] RBAC: Authorizing `identity:delete_project(project_id=863e2e49ffb74a889fb88c4c2e7edcd4)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:15.316692 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:15.568418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863e2e49ffb74a889fb88c4c2e7edcd4', 'request_id': 'req-1347ba41-03fc-436b-8b50-39394c51518e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:15.575128 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863e2e49ffb74a889fb88c4c2e7edcd4', 'request_id': 'req-1347ba41-03fc-436b-8b50-39394c51518e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:15.666865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863e2e49ffb74a889fb88c4c2e7edcd4', 'request_id': 'req-1347ba41-03fc-436b-8b50-39394c51518e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:15.678935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] The token cache is being invalidate because project 863e2e49ffb74a889fb88c4c2e7edcd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:00:15.679230 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:15.679899 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:15.681548 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1379/2741] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:00:15 2022] DELETE /identity/v3/projects/863e2e49ffb74a889fb88c4c2e7edcd4 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:00:15.685676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:15.687245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:15.701416 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba787d-bdbf-41c9-a52c-0bc8eb625caa admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:15.759002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:15.759801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:15.760088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:15.760369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] PATH_INFO: `/v3/users/a59db19c004e4f35b930fc4739cd2274` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:15.761676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59db19c004e4f35b930fc4739cd2274)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:15.763558 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:15.765878 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:00:16.079800 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59db19c004e4f35b930fc4739cd2274'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:16.091990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59db19c004e4f35b930fc4739cd2274'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:16.092693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59db19c004e4f35b930fc4739cd2274'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:16.093394 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59db19c004e4f35b930fc4739cd2274'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:16.101177 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-8c20d37b-2ee6-499f-86bf-f49c273e40e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59db19c004e4f35b930fc4739cd2274'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:16.220621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1363/2742] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:00:15 2022] DELETE /identity/v3/users/a59db19c004e4f35b930fc4739cd2274 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 18:00:16.519631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:16.521684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:16.537353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ba41-03fc-436b-8b50-39394c51518e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:16.565889 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:00:16.605044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:16.605918 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:16.606204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:16.606481 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] PATH_INFO: `/v3/projects/77aec6c344774102b7f54cff8e9667e0` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:16.607677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77aec6c344774102b7f54cff8e9667e0)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:16.609691 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:16.767302 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77aec6c344774102b7f54cff8e9667e0', 'request_id': 'req-5ca73698-2b2c-489a-a71f-53fff053eea9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:16.775314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77aec6c344774102b7f54cff8e9667e0', 'request_id': 'req-5ca73698-2b2c-489a-a71f-53fff053eea9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:17.003411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77aec6c344774102b7f54cff8e9667e0', 'request_id': 'req-5ca73698-2b2c-489a-a71f-53fff053eea9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:17.018301 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] The token cache is being invalidate because project 77aec6c344774102b7f54cff8e9667e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:00:17.024584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:17.024584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:00:17.024584 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1380/2743] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:00:16 2022] DELETE /identity/v3/projects/77aec6c344774102b7f54cff8e9667e0 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:00:18.719908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:18.720491 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:18.720623 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:18.741925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] MFA Rules not processed for user `de1ce4a75440478d957a3a03d41dbffd`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:00:18.743231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:18.801573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1364/2744] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:00:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:00:19.236363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:19.238233 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:19.325490 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca73698-2b2c-489a-a71f-53fff053eea9 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:19.334585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:19.335283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:19.335572 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:19.335860 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:19.337773 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:19.352600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:19.354120 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:19.367525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1381/2745] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:00:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:00:19.371830 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:19.373290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:19.435920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347c86a-e1d0-4fa1-bf85-faa7df918077 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:19.439613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:19.440283 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:19.440633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:19.440919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:19.442802 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:19.453893 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:19.454938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:19.467419 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1365/2746] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:00:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:00:20.113859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:20.115597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:20.170697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:00:20.198826 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7650ba9-f40d-498f-ae6f-e9bc37e6d994 service nova] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:20.204426 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:20.205651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:20.205953 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:20.206248 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:20.208229 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:20.225165 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:20.236930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:20.239995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1382/2747] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:00:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:00:29.239677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:00:29.249954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:29.257717 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bed6a1-bfdb-431f-bdc0-5b86433b0043 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:00:29.263352 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:00:29.264051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:00:29.264343 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:00:29.264649 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:00:29.266424 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:00:29.279251 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:00:29.280988 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:00:29.292333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1366/2748] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:00:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:00:31.254357 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=14 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:00:39.287567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=82,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:01:51.190970 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:51.192467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:51.259540 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d20e-87a7-430b-b21f-a222043f7192 service placement] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:51.264075 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:51.267220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:51.267220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:51.267220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] PATH_INFO: `/v3/users/de1ce4a75440478d957a3a03d41dbffd` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:51.267220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1ce4a75440478d957a3a03d41dbffd)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:51.269327 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:51.527085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1ce4a75440478d957a3a03d41dbffd'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:51.527085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1ce4a75440478d957a3a03d41dbffd'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:51.527085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1ce4a75440478d957a3a03d41dbffd'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:51.532631 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1ce4a75440478d957a3a03d41dbffd'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:51.664306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8a638a76-8a32-48a2-84c4-899e8cdadce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1ce4a75440478d957a3a03d41dbffd'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:51.681895 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1383/2749] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:01:51 2022] DELETE /identity/v3/users/de1ce4a75440478d957a3a03d41dbffd => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:01:51.957295 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:51.969984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608bfc-c68d-4004-82b2-98d7b5ad8d6b service nova] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:52.026939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:52.027790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.028073 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.028348 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] PATH_INFO: `/v3/projects/958d3fd7755a4e0b8419e92a26656e2d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.029669 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=958d3fd7755a4e0b8419e92a26656e2d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:52.031673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:52.155820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958d3fd7755a4e0b8419e92a26656e2d', 'request_id': 'req-443a1509-8ddc-4d0c-92f2-120258dc75b0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:52.161788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958d3fd7755a4e0b8419e92a26656e2d', 'request_id': 'req-443a1509-8ddc-4d0c-92f2-120258dc75b0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:52.167881 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958d3fd7755a4e0b8419e92a26656e2d', 'request_id': 'req-443a1509-8ddc-4d0c-92f2-120258dc75b0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:52.252077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] The token cache is being invalidate because project 958d3fd7755a4e0b8419e92a26656e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:01:52.252370 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:52.253018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-443a1509-8ddc-4d0c-92f2-120258dc75b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:52.254586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1367/2750] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:01:51 2022] DELETE /identity/v3/projects/958d3fd7755a4e0b8419e92a26656e2d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:01:52.259498 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65b828-e804-4c25-88a5-04a2352b19fd None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.259995 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65b828-e804-4c25-88a5-04a2352b19fd None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.267182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65b828-e804-4c25-88a5-04a2352b19fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.277880 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-ee65b828-e804-4c25-88a5-04a2352b19fd None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:01:52.279110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ee65b828-e804-4c25-88a5-04a2352b19fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:52.337006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1384/2751] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:01:52.344781 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa27e1-4f06-4309-9344-c0765e882fe8 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.345105 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa27e1-4f06-4309-9344-c0765e882fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.345382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa27e1-4f06-4309-9344-c0765e882fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.364213 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-52aa27e1-4f06-4309-9344-c0765e882fe8 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:01:52.365445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-52aa27e1-4f06-4309-9344-c0765e882fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:52.373310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1368/2752] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:01:52.377298 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65b828-e804-4c25-88a5-04a2352b19fd None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:52.386954 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65b828-e804-4c25-88a5-04a2352b19fd None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:52.391660 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:52.392312 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.392652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.392934 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.393689 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 18:01:52.393999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:52.395592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:52.406982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1385/2753] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 18:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 18:01:52.411999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa27e1-4f06-4309-9344-c0765e882fe8 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:52.421165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa27e1-4f06-4309-9344-c0765e882fe8 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:52.426008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:52.426825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.427654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.428012 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.428975 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:52.430963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:52.545733 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1369/2754] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:01:52 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 18:01:52.549962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:52.559876 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45a1e4-1cba-4b33-ba2c-f105569a2d39 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:52.565275 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:01:52.619567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:52.620160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.620477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.620760 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.621477 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:52.623430 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:52.750835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1386/2755] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:01:52 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:01:52.756166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:52.766568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b40605-10a2-4fcd-940c-f7e3f54fb8a3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:52.771655 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:52.772311 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.772631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.772914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.773778 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:52.775377 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:52.786341 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1370/2756] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:01:52.790325 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:52.800171 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11e3fb-8b62-4689-b83e-db130915457e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:52.804846 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551825f5-6016-4fe1-93e0-0bf77fa65b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:52.805499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551825f5-6016-4fe1-93e0-0bf77fa65b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.805785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551825f5-6016-4fe1-93e0-0bf77fa65b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.806063 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551825f5-6016-4fe1-93e0-0bf77fa65b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.806782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551825f5-6016-4fe1-93e0-0bf77fa65b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:52.808364 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551825f5-6016-4fe1-93e0-0bf77fa65b13 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:52.818703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1387/2757] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:01:52.822863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:52.833707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555206d-ca7d-4b2f-b6cb-e1144ae0956e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:52.838227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:52.839138 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.839428 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.839703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] PATH_INFO: `/v3/projects/b12411017bf54dee93b9d5a601db4283/users/6b8baf4ee05e4ac6b9cad3e7ec85b46e/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:52.854597 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b12411017bf54dee93b9d5a601db4283, user_id=6b8baf4ee05e4ac6b9cad3e7ec85b46e, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:52.857548 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:52.979379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1371/2758] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:01:52 2022] PUT /identity/v3/projects/b12411017bf54dee93b9d5a601db4283/users/6b8baf4ee05e4ac6b9cad3e7ec85b46e/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:01:52.990409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7bcfa-5f9f-4f28-9ad2-d92157fff4a9 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:52.990703 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7bcfa-5f9f-4f28-9ad2-d92157fff4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:52.990985 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7bcfa-5f9f-4f28-9ad2-d92157fff4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.017409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-1cc7bcfa-5f9f-4f28-9ad2-d92157fff4a9 None None] MFA Rules not processed for user `6b8baf4ee05e4ac6b9cad3e7ec85b46e`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:01:53.019148 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1cc7bcfa-5f9f-4f28-9ad2-d92157fff4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:53.082094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1388/2759] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:01:53.086591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:53.106952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:01:53.149303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857daef-2b65-41ce-84c9-1de9c9d41fcd admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:53.153920 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:53.154577 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.154860 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.155139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.155828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:53.157770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:53.260282 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1372/2760] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:01:53 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:01:53.264373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7bcfa-5f9f-4f28-9ad2-d92157fff4a9 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:53.275130 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7bcfa-5f9f-4f28-9ad2-d92157fff4a9 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:53.330975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:53.331758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.332055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.332338 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.333071 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:53.334987 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:53.431222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1389/2761] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:01:53 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:01:53.435302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:53.446158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98abb0-2e27-49d0-913c-3449e6a032e5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:53.450959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:53.451614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.451916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.452172 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.452929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:53.454519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:53.465180 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1373/2762] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:01:53.469432 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:53.475841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:01:53.479784 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f537e-b508-4261-b94b-d65ac81768ca admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:53.484306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:53.485204 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.485527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.485806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] PATH_INFO: `/v3/projects/e5cdf19e65d541c1a7102129f48479ef/users/54c48e3fe6dd4b2b9d921b08339a025c/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.501404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5cdf19e65d541c1a7102129f48479ef, user_id=54c48e3fe6dd4b2b9d921b08339a025c, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:53.504821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:53.579070 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1390/2763] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:01:53 2022] PUT /identity/v3/projects/e5cdf19e65d541c1a7102129f48479ef/users/54c48e3fe6dd4b2b9d921b08339a025c/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 18:01:53.582989 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:53.593001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d83f5-ee5a-425e-8c7a-0e1e0bfaec51 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:53.649753 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c9605-2333-4228-abce-87dea9c31fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:53.650417 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9605-2333-4228-abce-87dea9c31fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.650700 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9605-2333-4228-abce-87dea9c31fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.650974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9605-2333-4228-abce-87dea9c31fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.651707 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c9605-2333-4228-abce-87dea9c31fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:53.653396 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c9605-2333-4228-abce-87dea9c31fa9 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:53.664478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1374/2764] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:01:53.669570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:53.682150 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925613-0810-4cf4-9436-038d6a574d0c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:53.688289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:53.689228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.689510 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.689811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] PATH_INFO: `/v3/projects/e5cdf19e65d541c1a7102129f48479ef/users/54c48e3fe6dd4b2b9d921b08339a025c/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.691843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5cdf19e65d541c1a7102129f48479ef, user_id=54c48e3fe6dd4b2b9d921b08339a025c, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:53.694514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:53.705044 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1391/2765] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:01:53 2022] PUT /identity/v3/projects/e5cdf19e65d541c1a7102129f48479ef/users/54c48e3fe6dd4b2b9d921b08339a025c/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:01:53.715236 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2fb60-bcff-4573-9000-d05912a1fc20 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.715528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2fb60-bcff-4573-9000-d05912a1fc20 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.715811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2fb60-bcff-4573-9000-d05912a1fc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.742565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1ad2fb60-bcff-4573-9000-d05912a1fc20 None None] MFA Rules not processed for user `54c48e3fe6dd4b2b9d921b08339a025c`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:01:53.743842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1ad2fb60-bcff-4573-9000-d05912a1fc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:53.812995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1375/2766] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:01:53.824657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:53.826351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:53.886569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169a5ca-4f9b-41d1-84c5-774835428f1a admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:53.890213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:53.890857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:53.891205 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:53.891504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:53.901241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:53.902707 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:53.912739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1392/2767] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:01:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:01:54.169797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2fb60-bcff-4573-9000-d05912a1fc20 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:54.179666 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2fb60-bcff-4573-9000-d05912a1fc20 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:54.183299 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c02f2-0c27-49f9-a8dc-f475d60b981f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:54.183995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c02f2-0c27-49f9-a8dc-f475d60b981f service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:54.184277 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c02f2-0c27-49f9-a8dc-f475d60b981f service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:54.184585 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c02f2-0c27-49f9-a8dc-f475d60b981f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:54.194499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:01:54.241727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c02f2-0c27-49f9-a8dc-f475d60b981f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:54.243229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c02f2-0c27-49f9-a8dc-f475d60b981f service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:54.252651 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1376/2768] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:01:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:01:58.004222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:58.065226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971e66-9836-4c5f-8ed0-09889949335e service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:58.069785 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:58.070644 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:58.075797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:58.075797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] PATH_INFO: `/v3/users/6b8baf4ee05e4ac6b9cad3e7ec85b46e` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:58.075797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8baf4ee05e4ac6b9cad3e7ec85b46e)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:58.075797 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:58.272423 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8baf4ee05e4ac6b9cad3e7ec85b46e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:58.273129 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8baf4ee05e4ac6b9cad3e7ec85b46e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:58.273733 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8baf4ee05e4ac6b9cad3e7ec85b46e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:58.281200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8baf4ee05e4ac6b9cad3e7ec85b46e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:58.366727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8baf4ee05e4ac6b9cad3e7ec85b46e'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:58.374097 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:01:58.385373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1393/2769] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:01:58 2022] DELETE /identity/v3/users/6b8baf4ee05e4ac6b9cad3e7ec85b46e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:01:58.390132 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67672b8-ffbd-4a6e-8cbf-8499bad9d484 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:58.390595 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67672b8-ffbd-4a6e-8cbf-8499bad9d484 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:58.390709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67672b8-ffbd-4a6e-8cbf-8499bad9d484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:58.408625 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-a67672b8-ffbd-4a6e-8cbf-8499bad9d484 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:01:58.409832 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-a67672b8-ffbd-4a6e-8cbf-8499bad9d484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:58.467801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1377/2770] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 18:01:58.477533 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:58.479022 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:58.541975 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce5b3-52f0-41c7-a1b8-9fc12bc268f0 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:58.545553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:58.546192 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:58.546475 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:58.546751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:58.557582 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:58.560499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:58.569011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1394/2771] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:01:58.901084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a59a79-22e6-4127-a5e8-c093ff33d020 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:58.901411 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a59a79-22e6-4127-a5e8-c093ff33d020 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:58.901684 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a59a79-22e6-4127-a5e8-c093ff33d020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:58.919954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-40a59a79-22e6-4127-a5e8-c093ff33d020 None None] MFA Rules not processed for user `9c95e9df33ad4f819f26fb8b695df7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:01:58.924974 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-40a59a79-22e6-4127-a5e8-c093ff33d020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:58.980044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1378/2772] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:01:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:01:59.009461 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:59.011684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:01:59.029622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a19d-3c91-432e-b92f-76ec5af876b7 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:59.035683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:59.036780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:59.037442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:59.037890 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] PATH_INFO: `/v3/projects/b12411017bf54dee93b9d5a601db4283` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:59.039612 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b12411017bf54dee93b9d5a601db4283)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:59.045265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:59.175571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b12411017bf54dee93b9d5a601db4283', 'request_id': 'req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.181815 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b12411017bf54dee93b9d5a601db4283', 'request_id': 'req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.287806 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b12411017bf54dee93b9d5a601db4283', 'request_id': 'req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.299178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] The token cache is being invalidate because project b12411017bf54dee93b9d5a601db4283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:01:59.299544 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.300259 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.302074 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1395/2773] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:01:59 2022] DELETE /identity/v3/projects/b12411017bf54dee93b9d5a601db4283 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:01:59.306077 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a59a79-22e6-4127-a5e8-c093ff33d020 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:01:59.317245 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a59a79-22e6-4127-a5e8-c093ff33d020 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:01:59.370900 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:01:59.376605 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:01:59.377513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:01:59.377795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:01:59.378094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] PATH_INFO: `/v3/users/54c48e3fe6dd4b2b9d921b08339a025c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:01:59.379288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c48e3fe6dd4b2b9d921b08339a025c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:01:59.381268 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:01:59.581709 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c48e3fe6dd4b2b9d921b08339a025c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.594590 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c48e3fe6dd4b2b9d921b08339a025c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.595510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c48e3fe6dd4b2b9d921b08339a025c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.596114 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c48e3fe6dd4b2b9d921b08339a025c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.603471 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c48e3fe6dd4b2b9d921b08339a025c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:01:59.681560 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1379/2774] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:01:59 2022] DELETE /identity/v3/users/54c48e3fe6dd4b2b9d921b08339a025c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:00.106622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:00.106622 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:00.109382 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:00.111318 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:00.195621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d87124-1e62-4734-a6e7-6035057792a6 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:00.200373 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:00.201272 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.201559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:00.202024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] PATH_INFO: `/v3/projects/e5cdf19e65d541c1a7102129f48479ef` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:00.203072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5cdf19e65d541c1a7102129f48479ef)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:00.205339 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a03b7-32fa-4f1f-af52-cb6cc629bea6 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:00.205629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:00.216647 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:00.275709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92290c2-9487-4beb-b310-d6ab1f921f32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6f536917f434773a052aa074d5aa07d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:00.276366 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92290c2-9487-4beb-b310-d6ab1f921f32 service nova] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.276684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92290c2-9487-4beb-b310-d6ab1f921f32 service nova] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:00.276962 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92290c2-9487-4beb-b310-d6ab1f921f32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:00.278728 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-b92290c2-9487-4beb-b310-d6ab1f921f32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:00.334614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5cdf19e65d541c1a7102129f48479ef', 'request_id': 'req-6a96049e-b689-4a67-81bd-85642f75c0b0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:00.339542 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92290c2-9487-4beb-b310-d6ab1f921f32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:00.340993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5cdf19e65d541c1a7102129f48479ef', 'request_id': 'req-6a96049e-b689-4a67-81bd-85642f75c0b0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:00.341208 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92290c2-9487-4beb-b310-d6ab1f921f32 service nova] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:00.347571 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5cdf19e65d541c1a7102129f48479ef', 'request_id': 'req-6a96049e-b689-4a67-81bd-85642f75c0b0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:00.351178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1396/2775] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:00.481497 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] The token cache is being invalidate because project e5cdf19e65d541c1a7102129f48479ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:02:00.481792 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:00.486789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-6a96049e-b689-4a67-81bd-85642f75c0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:00.486789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1380/2776] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:02:00 2022] DELETE /identity/v3/projects/e5cdf19e65d541c1a7102129f48479ef => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 18:02:00.488822 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9611c-7e88-4959-9e7f-931ed2f4803b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.489124 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9611c-7e88-4959-9e7f-931ed2f4803b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:00.489412 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9611c-7e88-4959-9e7f-931ed2f4803b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:00.507687 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-46b9611c-7e88-4959-9e7f-931ed2f4803b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:00.508951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-46b9611c-7e88-4959-9e7f-931ed2f4803b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:00.566870 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1397/2777] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:00.574791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747ccb4-6cf9-428d-aa1a-c44c29973c45 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.574908 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747ccb4-6cf9-428d-aa1a-c44c29973c45 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:00.575194 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747ccb4-6cf9-428d-aa1a-c44c29973c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:00.588400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:00.594490 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-4747ccb4-6cf9-428d-aa1a-c44c29973c45 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:00.596713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-4747ccb4-6cf9-428d-aa1a-c44c29973c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:00.604045 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1381/2778] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:00.608575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9611c-7e88-4959-9e7f-931ed2f4803b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:00.618679 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9611c-7e88-4959-9e7f-931ed2f4803b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:00.624268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:00.624268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.624268 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:00.624678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:00.625282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 18:02:00.625597 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:00.627709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:00.639280 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1398/2779] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 18:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 18:02:00.644221 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747ccb4-6cf9-428d-aa1a-c44c29973c45 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:00.653673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747ccb4-6cf9-428d-aa1a-c44c29973c45 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:00.658575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:00.659249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.659540 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:00.659822 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:00.660562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:00.662474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:00.774742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1382/2780] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:02:00 2022] POST /identity/v3/projects => generated 401 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:00.778978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:00.789141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056a146-988b-400b-8c27-cab2b279dc58 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:00.845821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:00.846519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.846809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:00.847088 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:00.847958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:00.849939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:00.979725 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1399/2781] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:02:00 2022] POST /identity/v3/users => generated 301 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 18:02:00.983926 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:00.985375 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:00.994537 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b022dea-d7e0-4ded-8531-bb4eba80172d admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:00.999026 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:00.999679 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:00.999962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.000237 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.001006 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.003331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:01.013803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1383/2782] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:01.017893 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:01.027841 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6469033-c2e2-427a-995c-ac842e5b7932 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:01.032390 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab3762-8240-4f8e-b5ae-5ec598ff6f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:01.033098 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3762-8240-4f8e-b5ae-5ec598ff6f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.033388 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3762-8240-4f8e-b5ae-5ec598ff6f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.033671 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3762-8240-4f8e-b5ae-5ec598ff6f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.034404 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab3762-8240-4f8e-b5ae-5ec598ff6f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.037638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab3762-8240-4f8e-b5ae-5ec598ff6f21 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:01.049980 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1400/2783] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:01.054223 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:01.064959 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef9737-3834-4205-aa88-8d10f6628bfc admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:01.069510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:01.070390 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.070704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.070982 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] PATH_INFO: `/v3/projects/f039de5bcf9b413480e107cc8949e6aa/users/15cd0e62595f4d629ab6bb2491c8a0e7/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.087053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f039de5bcf9b413480e107cc8949e6aa, user_id=15cd0e62595f4d629ab6bb2491c8a0e7, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.089909 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:01.179867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1384/2784] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:01 2022] PUT /identity/v3/projects/f039de5bcf9b413480e107cc8949e6aa/users/15cd0e62595f4d629ab6bb2491c8a0e7/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 18:02:01.191491 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6de45-6ef5-4ca1-a0d5-5b67bf624c6d None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.192042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6de45-6ef5-4ca1-a0d5-5b67bf624c6d None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.192042 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6de45-6ef5-4ca1-a0d5-5b67bf624c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.220270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-10a6de45-6ef5-4ca1-a0d5-5b67bf624c6d None None] MFA Rules not processed for user `15cd0e62595f4d629ab6bb2491c8a0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:01.222135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-10a6de45-6ef5-4ca1-a0d5-5b67bf624c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:01.285309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1401/2785] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:01.289787 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:01.351181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf2950-180c-4456-b7f2-65089ff4d0bd admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:01.355984 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:01.356673 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.356953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.369004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.369004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.369004 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:01.500561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1385/2786] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:02:01 2022] POST /identity/v3/projects => generated 401 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:01.504656 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6de45-6ef5-4ca1-a0d5-5b67bf624c6d None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:01.515567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6de45-6ef5-4ca1-a0d5-5b67bf624c6d None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:01.574898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:01.575499 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.575789 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.576072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.576803 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.578739 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:01.700508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1402/2787] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:02:01 2022] POST /identity/v3/users => generated 307 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:01.704659 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:01.715813 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1867088-7899-451e-bdf3-65028aae8e97 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:01.722883 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:01.723568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.723868 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.724136 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.724952 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.726641 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:01.737867 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1386/2788] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:01.741782 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:01.751685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acecac-7a47-466f-85ea-cca18369a3f2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:01.756878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:01.757774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.758060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.758342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] PATH_INFO: `/v3/projects/ddfb0fd708724bfa9c0a28a424987b9d/users/5454402aed4446ef94054a4c77e68f82/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.774669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddfb0fd708724bfa9c0a28a424987b9d, user_id=5454402aed4446ef94054a4c77e68f82, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.777363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:01.921825 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1403/2789] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:01 2022] PUT /identity/v3/projects/ddfb0fd708724bfa9c0a28a424987b9d/users/5454402aed4446ef94054a4c77e68f82/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:01.925925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:01.936151 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacda30-848a-4d73-8a8a-70b330e490f2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:01.996092 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca9fef-fe59-4c5e-abb4-e4d2b58f1b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:01.996788 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca9fef-fe59-4c5e-abb4-e4d2b58f1b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:01.997075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca9fef-fe59-4c5e-abb4-e4d2b58f1b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:01.997351 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca9fef-fe59-4c5e-abb4-e4d2b58f1b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:01.998076 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca9fef-fe59-4c5e-abb4-e4d2b58f1b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:01.999704 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca9fef-fe59-4c5e-abb4-e4d2b58f1b86 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:02.013086 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1387/2790] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:02.017101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:02.028632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a9feb-20f9-475f-bf1d-36af33b0cf4c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:02.040981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:02.040981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:02.040981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:02.040981 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] PATH_INFO: `/v3/projects/ddfb0fd708724bfa9c0a28a424987b9d/users/5454402aed4446ef94054a4c77e68f82/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:02.041592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddfb0fd708724bfa9c0a28a424987b9d, user_id=5454402aed4446ef94054a4c77e68f82, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:02.041592 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:02.046003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:02.051974 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1404/2791] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:02 2022] PUT /identity/v3/projects/ddfb0fd708724bfa9c0a28a424987b9d/users/5454402aed4446ef94054a4c77e68f82/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:02.063020 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5ad8d-6efc-435e-b9a9-cdea0889c847 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:02.063307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5ad8d-6efc-435e-b9a9-cdea0889c847 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:02.063587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5ad8d-6efc-435e-b9a9-cdea0889c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:02.090095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:02.092991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-44e5ad8d-6efc-435e-b9a9-cdea0889c847 None None] MFA Rules not processed for user `5454402aed4446ef94054a4c77e68f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:02.094253 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-44e5ad8d-6efc-435e-b9a9-cdea0889c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:02.165029 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1388/2792] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:02.176547 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:02.178115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:02.234036 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adec017-7fcc-434b-a88c-8b748d3838e3 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:02.237651 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:02.238284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:02.238569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:02.238844 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:02.248277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:02.249864 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:02.259380 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1405/2793] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:02.495266 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5ad8d-6efc-435e-b9a9-cdea0889c847 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:02.505229 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5ad8d-6efc-435e-b9a9-cdea0889c847 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:02.508871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061092f-3541-4dc1-89dc-177f82830c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:02.509518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061092f-3541-4dc1-89dc-177f82830c79 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:02.509809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061092f-3541-4dc1-89dc-177f82830c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:02.510084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061092f-3541-4dc1-89dc-177f82830c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:02.578793 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061092f-3541-4dc1-89dc-177f82830c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:02.580928 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061092f-3541-4dc1-89dc-177f82830c79 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:02.590212 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1389/2794] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:07.786824 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:07.854260 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f3111-c177-4536-bbc0-823fed1adab1 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:07.860093 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:07.861031 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:07.861834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:07.862113 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] PATH_INFO: `/v3/users/15cd0e62595f4d629ab6bb2491c8a0e7` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:07.865224 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] RBAC: Authorizing `identity:delete_user(user_id=15cd0e62595f4d629ab6bb2491c8a0e7)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:07.867160 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:08.107908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15cd0e62595f4d629ab6bb2491c8a0e7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:08.108587 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15cd0e62595f4d629ab6bb2491c8a0e7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:08.109199 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15cd0e62595f4d629ab6bb2491c8a0e7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:08.116303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15cd0e62595f4d629ab6bb2491c8a0e7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:08.195630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15cd0e62595f4d629ab6bb2491c8a0e7'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:08.214238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1406/2795] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:02:07 2022] DELETE /identity/v3/users/15cd0e62595f4d629ab6bb2491c8a0e7 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:08.218801 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:08.219325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:08.219437 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:08.236565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:08.237764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:08.301102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1390/2796] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:08.310744 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:08.312175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:08.334405 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:08.369081 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641b41a-4600-4e33-9b12-83b49b18e11c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:08.373084 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:08.374100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:08.374452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:08.374737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:08.384359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:08.385843 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:08.395346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1407/2797] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:08.886362 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:08.889780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:08.901924 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d523-33d3-426d-8703-8f87fd94af9d None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:08.907482 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:08.908941 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:08.909225 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:08.909508 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] PATH_INFO: `/v3/projects/f039de5bcf9b413480e107cc8949e6aa` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:08.910782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f039de5bcf9b413480e107cc8949e6aa)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:08.912739 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:09.041999 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:09.053454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f039de5bcf9b413480e107cc8949e6aa', 'request_id': 'req-0373de5e-0170-413e-933d-a9b238b53d6e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.058799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f039de5bcf9b413480e107cc8949e6aa', 'request_id': 'req-0373de5e-0170-413e-933d-a9b238b53d6e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.065144 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f039de5bcf9b413480e107cc8949e6aa', 'request_id': 'req-0373de5e-0170-413e-933d-a9b238b53d6e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.180289 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] The token cache is being invalidate because project f039de5bcf9b413480e107cc8949e6aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:02:09.180624 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.181254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.182845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1391/2798] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:02:08 2022] DELETE /identity/v3/projects/f039de5bcf9b413480e107cc8949e6aa => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:09.187017 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:09.199246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa80c79-46c9-4be7-b431-fd4114b6857b service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:09.260175 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:09.261140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:09.261428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:09.261786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] PATH_INFO: `/v3/users/5454402aed4446ef94054a4c77e68f82` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:09.263006 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5454402aed4446ef94054a4c77e68f82)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:09.265141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:09.438500 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5454402aed4446ef94054a4c77e68f82'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.439752 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5454402aed4446ef94054a4c77e68f82'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.440355 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5454402aed4446ef94054a4c77e68f82'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.447439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5454402aed4446ef94054a4c77e68f82'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.538629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-aaf6ef9f-4916-4154-9bc8-57d8baa2b2ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5454402aed4446ef94054a4c77e68f82'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:09.557517 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1408/2799] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:02:09 2022] DELETE /identity/v3/users/5454402aed4446ef94054a4c77e68f82 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:10.009063 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:10.010554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:10.024261 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373de5e-0170-413e-933d-a9b238b53d6e admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:10.083818 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:10.084722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.085008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.085286 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] PATH_INFO: `/v3/projects/ddfb0fd708724bfa9c0a28a424987b9d` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.086600 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddfb0fd708724bfa9c0a28a424987b9d)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:10.088584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:10.209706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddfb0fd708724bfa9c0a28a424987b9d', 'request_id': 'req-4968d535-d83c-4aa5-8928-c93c02b4cb86'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:10.215192 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddfb0fd708724bfa9c0a28a424987b9d', 'request_id': 'req-4968d535-d83c-4aa5-8928-c93c02b4cb86'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:10.221575 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddfb0fd708724bfa9c0a28a424987b9d', 'request_id': 'req-4968d535-d83c-4aa5-8928-c93c02b4cb86'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:10.349591 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] The token cache is being invalidate because project ddfb0fd708724bfa9c0a28a424987b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:02:10.355564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:10.355564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-4968d535-d83c-4aa5-8928-c93c02b4cb86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:10.355564 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1392/2800] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:02:10 2022] DELETE /identity/v3/projects/ddfb0fd708724bfa9c0a28a424987b9d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:10.357169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cdc41-71db-46b4-aaf0-3c55402e5482 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.357623 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cdc41-71db-46b4-aaf0-3c55402e5482 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.357740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cdc41-71db-46b4-aaf0-3c55402e5482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.375360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-8b1cdc41-71db-46b4-aaf0-3c55402e5482 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:10.377191 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8b1cdc41-71db-46b4-aaf0-3c55402e5482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:10.415559 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:10.436207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1409/2801] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:10.445650 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd64e94-82d0-4429-b4a1-19056cd4e425 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.452518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd64e94-82d0-4429-b4a1-19056cd4e425 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.452518 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd64e94-82d0-4429-b4a1-19056cd4e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.464948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-8cd64e94-82d0-4429-b4a1-19056cd4e425 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:10.466159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-8cd64e94-82d0-4429-b4a1-19056cd4e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:10.474276 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1393/2802] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:10.478410 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cdc41-71db-46b4-aaf0-3c55402e5482 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:10.488603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cdc41-71db-46b4-aaf0-3c55402e5482 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:10.493207 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:10.503957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.503957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.503957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.503957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 18:02:10.503957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:10.503957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:10.509213 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1410/2803] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 18:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 18:02:10.513302 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd64e94-82d0-4429-b4a1-19056cd4e425 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:10.522768 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd64e94-82d0-4429-b4a1-19056cd4e425 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:10.527544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:10.528190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.528510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.528794 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.529489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:10.531440 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:10.682921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1394/2804] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:02:10 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:10.687032 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:10.697944 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1b5fb-1476-4c8c-9ea9-13f1f677babd admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:10.755078 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:10.755731 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.756066 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.756350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.757072 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:10.759011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:10.893105 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1411/2805] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:02:10 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:10.896976 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:10.907414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae76f6-1146-47b9-adc9-ae7f5a3e5a27 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:10.912476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:10.913130 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.913413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.913692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.914445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:10.916031 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:10.926785 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1395/2806] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:10.930601 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:10.941538 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c18263-d2d2-48f0-af39-09ecb402aadb admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:10.945514 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65828c-4551-45c7-b023-78b85f131c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:10.946585 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65828c-4551-45c7-b023-78b85f131c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.946873 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65828c-4551-45c7-b023-78b85f131c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.947154 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65828c-4551-45c7-b023-78b85f131c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.948139 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65828c-4551-45c7-b023-78b85f131c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:10.949766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65828c-4551-45c7-b023-78b85f131c7c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:10.959553 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1412/2807] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:10.963721 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:10.974593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f6d05-660a-42f5-983e-f59bd18c99d7 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:10.980153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:10.981111 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:10.981400 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:10.981680 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] PATH_INFO: `/v3/projects/e467c83d7197474d9943d6861971f10a/users/a27cd430ab6749c085c76ad54c64732c/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:10.990897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:10.998886 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e467c83d7197474d9943d6861971f10a, user_id=a27cd430ab6749c085c76ad54c64732c, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:11.001592 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:11.162215 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1396/2808] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:10 2022] PUT /identity/v3/projects/e467c83d7197474d9943d6861971f10a/users/a27cd430ab6749c085c76ad54c64732c/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:11.172630 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83832cc9-63d0-4e36-a8b0-b3bbda776ab5 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:11.172926 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83832cc9-63d0-4e36-a8b0-b3bbda776ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:11.173267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83832cc9-63d0-4e36-a8b0-b3bbda776ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:11.200061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-83832cc9-63d0-4e36-a8b0-b3bbda776ab5 None None] MFA Rules not processed for user `a27cd430ab6749c085c76ad54c64732c`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:11.201122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-83832cc9-63d0-4e36-a8b0-b3bbda776ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:11.265424 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1413/2809] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:11.269938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:11.464799 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e1309-98d4-4485-91bc-c166b6a88471 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:11.469254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:11.469904 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:11.470201 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:11.470504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:11.471181 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:11.473099 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:11.654555 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1397/2810] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:02:11 2022] POST /identity/v3/projects => generated 381 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:11.659064 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83832cc9-63d0-4e36-a8b0-b3bbda776ab5 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:11.669779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83832cc9-63d0-4e36-a8b0-b3bbda776ab5 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:11.703398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:11.725967 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:11.729091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:11.729091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:11.729091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:11.729091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:11.729837 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:11.825990 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1414/2811] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:02:11 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:11.829971 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:11.840970 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab2c75-c7b7-48ff-a43e-ed6889ede6a3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:11.847055 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:11.847727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:11.848028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:11.848306 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:11.849070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:11.850779 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:11.861983 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1398/2812] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:11.866291 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:11.876618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98841d9-08b0-4e88-9918-edacc984c1a2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:11.881469 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:11.882376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:11.882668 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:11.882951 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] PATH_INFO: `/v3/projects/859c17b3b39d43d583ee858584010197/users/9a6159fbdfbf4a3a9b69cd499691553e/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:11.898000 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] RBAC: Authorizing `identity:create_grant(project_id=859c17b3b39d43d583ee858584010197, user_id=9a6159fbdfbf4a3a9b69cd499691553e, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:11.900794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:12.025583 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1415/2813] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:11 2022] PUT /identity/v3/projects/859c17b3b39d43d583ee858584010197/users/9a6159fbdfbf4a3a9b69cd499691553e/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:12.029582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:12.039932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bd4b1-00aa-4534-9739-005c8ef154c4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:12.105938 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a32e0-289c-4649-8525-e289f2add98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:12.106636 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32e0-289c-4649-8525-e289f2add98f admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:12.106925 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32e0-289c-4649-8525-e289f2add98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:12.107204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a32e0-289c-4649-8525-e289f2add98f admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:12.107948 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a32e0-289c-4649-8525-e289f2add98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:12.109713 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a32e0-289c-4649-8525-e289f2add98f admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:12.120631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1399/2814] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:12.124737 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:12.135569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e353b3-6dd9-4943-8784-ba564f3d234e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:12.161527 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:12.162719 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:12.163112 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:12.163504 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] PATH_INFO: `/v3/projects/859c17b3b39d43d583ee858584010197/users/9a6159fbdfbf4a3a9b69cd499691553e/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:12.165994 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=859c17b3b39d43d583ee858584010197, user_id=9a6159fbdfbf4a3a9b69cd499691553e, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:12.168982 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:12.189714 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1416/2815] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:12 2022] PUT /identity/v3/projects/859c17b3b39d43d583ee858584010197/users/9a6159fbdfbf4a3a9b69cd499691553e/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:12.191574 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed3b58-6d30-4b74-ba1f-f9a51c1f05c9 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:12.192034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed3b58-6d30-4b74-ba1f-f9a51c1f05c9 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:12.192165 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed3b58-6d30-4b74-ba1f-f9a51c1f05c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:12.221582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-81ed3b58-6d30-4b74-ba1f-f9a51c1f05c9 None None] MFA Rules not processed for user `9a6159fbdfbf4a3a9b69cd499691553e`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:12.222821 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-81ed3b58-6d30-4b74-ba1f-f9a51c1f05c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:12.277315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:12.301439 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1400/2816] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:12.312314 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:12.314645 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:12.372819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3dff0-beb9-44f2-9559-e28c677ef632 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:12.377628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d81743-9886-428a-a2a6-d09be3e1f9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:12.378287 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d81743-9886-428a-a2a6-d09be3e1f9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:12.378577 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d81743-9886-428a-a2a6-d09be3e1f9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:12.378857 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d81743-9886-428a-a2a6-d09be3e1f9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:12.389170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d81743-9886-428a-a2a6-d09be3e1f9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:12.390765 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d81743-9886-428a-a2a6-d09be3e1f9d5 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:12.401552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1417/2817] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:12.952910 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed3b58-6d30-4b74-ba1f-f9a51c1f05c9 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:13.014023 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed3b58-6d30-4b74-ba1f-f9a51c1f05c9 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:13.018593 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:13.019457 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:13.019742 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:13.020024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] PATH_INFO: `/v3/users/a27cd430ab6749c085c76ad54c64732c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:13.021239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a27cd430ab6749c085c76ad54c64732c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:13.023173 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:13.214953 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a27cd430ab6749c085c76ad54c64732c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:13.227561 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a27cd430ab6749c085c76ad54c64732c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:13.228249 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a27cd430ab6749c085c76ad54c64732c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:13.228895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a27cd430ab6749c085c76ad54c64732c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:13.235738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a27cd430ab6749c085c76ad54c64732c'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:13.315170 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1401/2818] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:02:12 2022] DELETE /identity/v3/users/a27cd430ab6749c085c76ad54c64732c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:13.326140 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:13.326505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:13.326696 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:13.344029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:13.345780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:13.405925 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1418/2819] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:13.416498 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:13.417980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:13.476049 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a921-4de9-4886-a9fc-402e4fe300c5 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:13.479805 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:13.480481 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:13.480770 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:13.481048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:13.490153 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:13.491582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:13.500838 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1402/2820] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:13.969381 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:13.971558 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:13.984009 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a087a0-2879-4801-bc58-3f36c13b141f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:13.988791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:13.989673 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:13.989957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:13.990231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] PATH_INFO: `/v3/projects/e467c83d7197474d9943d6861971f10a` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:13.991418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e467c83d7197474d9943d6861971f10a)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:13.993503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:14.112908 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:14.128418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e467c83d7197474d9943d6861971f10a', 'request_id': 'req-89600f06-790d-4a2b-8dce-eb424e57101f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.134935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e467c83d7197474d9943d6861971f10a', 'request_id': 'req-89600f06-790d-4a2b-8dce-eb424e57101f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.287522 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e467c83d7197474d9943d6861971f10a', 'request_id': 'req-89600f06-790d-4a2b-8dce-eb424e57101f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.300166 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] The token cache is being invalidate because project e467c83d7197474d9943d6861971f10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:02:14.300473 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.301103 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.302635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1419/2821] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:02:13 2022] DELETE /identity/v3/projects/e467c83d7197474d9943d6861971f10a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:14.306740 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:14.312783 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:14.317028 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81bc70-6352-4e42-bb1e-72ae3a368ef8 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:14.372933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:14.373806 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:14.374088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:14.374364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] PATH_INFO: `/v3/users/9a6159fbdfbf4a3a9b69cd499691553e` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:14.375549 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6159fbdfbf4a3a9b69cd499691553e)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:14.378158 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:14.626895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6159fbdfbf4a3a9b69cd499691553e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.641736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6159fbdfbf4a3a9b69cd499691553e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.644614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6159fbdfbf4a3a9b69cd499691553e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.644614 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6159fbdfbf4a3a9b69cd499691553e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.650648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c376267a-d0d6-4fac-b96d-b9a215e31b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6159fbdfbf4a3a9b69cd499691553e'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:14.728573 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1403/2822] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:02:14 2022] DELETE /identity/v3/users/9a6159fbdfbf4a3a9b69cd499691553e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:15.168778 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:15.170267 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:15.186274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89600f06-790d-4a2b-8dce-eb424e57101f admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:15.251179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:15.252056 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:15.252342 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:15.252661 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] PATH_INFO: `/v3/projects/859c17b3b39d43d583ee858584010197` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:15.253848 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=859c17b3b39d43d583ee858584010197)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:15.255795 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:15.365923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '859c17b3b39d43d583ee858584010197', 'request_id': 'req-e4d9b93e-c18c-45fd-a315-c08d62e00a48'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:15.371877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '859c17b3b39d43d583ee858584010197', 'request_id': 'req-e4d9b93e-c18c-45fd-a315-c08d62e00a48'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:15.463973 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '859c17b3b39d43d583ee858584010197', 'request_id': 'req-e4d9b93e-c18c-45fd-a315-c08d62e00a48'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:15.474347 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] The token cache is being invalidate because project 859c17b3b39d43d583ee858584010197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:02:15.474632 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:15.475247 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-e4d9b93e-c18c-45fd-a315-c08d62e00a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:15.476810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1420/2823] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:02:15 2022] DELETE /identity/v3/projects/859c17b3b39d43d583ee858584010197 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:15.481767 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd29c-d5c6-475f-a372-842381b8bd14 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:15.482290 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd29c-d5c6-475f-a372-842381b8bd14 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:15.482420 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd29c-d5c6-475f-a372-842381b8bd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:15.500104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bfbdd29c-d5c6-475f-a372-842381b8bd14 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:15.501333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bfbdd29c-d5c6-475f-a372-842381b8bd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:15.559145 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1404/2824] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:15.621061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eda873-8611-4efb-9339-f6f9ac205e23 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:15.621359 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eda873-8611-4efb-9339-f6f9ac205e23 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:15.621635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eda873-8611-4efb-9339-f6f9ac205e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:15.640193 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-05eda873-8611-4efb-9339-f6f9ac205e23 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:15.641420 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-05eda873-8611-4efb-9339-f6f9ac205e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:15.649898 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1421/2825] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:15.654392 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd29c-d5c6-475f-a372-842381b8bd14 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:15.664756 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd29c-d5c6-475f-a372-842381b8bd14 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:15.669359 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:15.670005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:15.670288 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:15.670566 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:15.671292 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 18:02:15.671598 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:15.673230 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:15.685397 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1405/2826] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 18:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 18:02:15.696738 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eda873-8611-4efb-9339-f6f9ac205e23 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:15.707100 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eda873-8611-4efb-9339-f6f9ac205e23 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:15.711940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:15.712652 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:15.712943 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:15.713223 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:15.714029 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:15.715961 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:15.860107 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1422/2827] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:02:15 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:15.867467 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:15.877231 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ea9f8-9621-4b4b-bb10-9af628c7d5a2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:15.884676 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:15.931842 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:15.932529 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:15.932812 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:15.933084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:15.933763 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:15.935662 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:16.041798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1406/2828] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:02:15 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:16.046146 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:16.056258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a693d2b-36f7-477b-b2e1-a90af5dbfa39 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:16.061221 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:16.061884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:16.062189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:16.062455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:16.063184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:16.065480 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:16.077495 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1423/2829] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:16.082005 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:16.091995 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7442003-264e-4019-bb4a-08f3194b0ca4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:16.096565 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e448bf-5fd7-4db8-8fde-14805247bf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:16.097227 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e448bf-5fd7-4db8-8fde-14805247bf2d admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:16.097511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e448bf-5fd7-4db8-8fde-14805247bf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:16.097780 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e448bf-5fd7-4db8-8fde-14805247bf2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:16.098579 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e448bf-5fd7-4db8-8fde-14805247bf2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:16.100094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e448bf-5fd7-4db8-8fde-14805247bf2d admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:16.111321 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1407/2830] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:16.114520 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:16.124360 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a40ab4-0729-4a7d-a4b1-9dccd91cdf5d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:16.129562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:16.130442 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:16.130730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:16.131012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] PATH_INFO: `/v3/projects/38ede1d66d4549a7bf002db641fa11f2/users/7e114dc2ea54427285f383b99fe6fb3f/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:16.134618 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:16.268804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ede1d66d4549a7bf002db641fa11f2, user_id=7e114dc2ea54427285f383b99fe6fb3f, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:16.271435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:16.367082 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1424/2831] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:16 2022] PUT /identity/v3/projects/38ede1d66d4549a7bf002db641fa11f2/users/7e114dc2ea54427285f383b99fe6fb3f/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:16.372044 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e269970-e334-47a8-a179-4cbca3d755ac None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:16.372372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e269970-e334-47a8-a179-4cbca3d755ac None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:16.372675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e269970-e334-47a8-a179-4cbca3d755ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:16.390325 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1e269970-e334-47a8-a179-4cbca3d755ac None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:16.393053 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1e269970-e334-47a8-a179-4cbca3d755ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:16.449633 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1408/2832] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:16.459698 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:16.461384 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:16.516704 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1b950-0948-4b4b-b9b2-ee0543e1a76e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:16.520271 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:16.520940 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:16.521227 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:16.521505 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:16.531658 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:16.533324 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:16.542613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1425/2833] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:18.569072 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec0a9-169a-4690-99ad-771b24caeadb None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:18.569621 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec0a9-169a-4690-99ad-771b24caeadb None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:18.569834 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec0a9-169a-4690-99ad-771b24caeadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:18.598150 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b08ec0a9-169a-4690-99ad-771b24caeadb None None] MFA Rules not processed for user `7e114dc2ea54427285f383b99fe6fb3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:18.599990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b08ec0a9-169a-4690-99ad-771b24caeadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:18.670929 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1409/2834] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:18.675472 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:18.685033 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c42cb-cbf5-4eb9-a1e8-ae87e5c4f4da service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:18.690269 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:18.696340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:18.696340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:18.696340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:18.696340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:18.696340 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:18.791786 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1426/2835] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:02:18 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:18.795891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ec0a9-169a-4690-99ad-771b24caeadb None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:18.806917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ec0a9-169a-4690-99ad-771b24caeadb None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:18.823991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:18.862554 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:18.863314 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:18.863607 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:18.863885 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:18.864613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:18.866541 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:18.970568 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1410/2836] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:02:18 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:18.971909 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:18.982779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7b43-08bf-4916-b128-1a36be7b58f2 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:18.988206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:18.988904 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:18.989190 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:18.989467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:18.990200 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:18.991810 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:19.002923 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1427/2837] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:19.007096 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:19.017586 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97ce21-6195-4069-9d02-cf534c0799f2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:19.022128 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:19.023015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:19.023297 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:19.023578 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] PATH_INFO: `/v3/projects/7e23becce79241bd85f6b602a5e0b08f/users/39862ae401ac4891b524d6546dab4790/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:19.038671 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e23becce79241bd85f6b602a5e0b08f, user_id=39862ae401ac4891b524d6546dab4790, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:19.041484 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:19.122969 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1411/2838] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:19 2022] PUT /identity/v3/projects/7e23becce79241bd85f6b602a5e0b08f/users/39862ae401ac4891b524d6546dab4790/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:19.127633 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:19.138648 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026133b-45c2-4037-9ac4-b36e1265f64d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:19.199787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fbb57f-3668-4536-97f0-f2930f546424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:19.200606 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbb57f-3668-4536-97f0-f2930f546424 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:19.201110 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbb57f-3668-4536-97f0-f2930f546424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:19.201465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fbb57f-3668-4536-97f0-f2930f546424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:19.202270 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbb57f-3668-4536-97f0-f2930f546424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:19.203931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fbb57f-3668-4536-97f0-f2930f546424 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:19.214655 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1428/2839] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:19.218749 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:19.228916 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6b1ff-59a8-413a-b968-a42c5994f6c3 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:19.233556 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:19.234438 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:19.234727 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:19.235008 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] PATH_INFO: `/v3/projects/7e23becce79241bd85f6b602a5e0b08f/users/39862ae401ac4891b524d6546dab4790/roles/2f9c0373d6e7487d8f2f97663fe18ecf` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:19.237409 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e23becce79241bd85f6b602a5e0b08f, user_id=39862ae401ac4891b524d6546dab4790, role_id=2f9c0373d6e7487d8f2f97663fe18ecf)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:19.244094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:19.255663 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1412/2840] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:19 2022] PUT /identity/v3/projects/7e23becce79241bd85f6b602a5e0b08f/users/39862ae401ac4891b524d6546dab4790/roles/2f9c0373d6e7487d8f2f97663fe18ecf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 18:02:20.724114 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:20.724984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:20.725274 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:20.752756 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] MFA Rules not processed for user `39862ae401ac4891b524d6546dab4790`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:20.753978 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:20.792603 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:20.821804 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1429/2841] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:20.833606 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:20.835675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:20.899310 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52b714-5715-4456-aec2-ae76fba79b0b admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:20.902912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:20.903553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:20.903833 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:20.904106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:20.959781 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:20.961337 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:20.970814 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1413/2842] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:22.855232 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:22.856664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:22.868935 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e8f9c-d3a9-46f4-8cb6-cc6a7402e802 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:22.872487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:22.873122 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:22.873406 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:22.873684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:22.882931 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:22.884968 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:22.894018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1430/2843] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:30.966496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:32.889709 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=18 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:39.046152 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:39.047604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:39.119921 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfde596-ef37-4dbb-8781-d216d4d1e440 service cinder] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:39.124939 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:39.125587 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:39.125960 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:39.126255 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:39.128101 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:39.139534 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:39.141155 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:39.152037 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1414/2844] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:39.973701 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:40.035683 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69ac13-bdba-47e9-a920-b5c8f7fccdef service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:40.040289 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:40.041189 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:40.041476 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:40.041763 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] PATH_INFO: `/v3/users/7e114dc2ea54427285f383b99fe6fb3f` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:40.042960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e114dc2ea54427285f383b99fe6fb3f)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:40.044914 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:40.329197 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e114dc2ea54427285f383b99fe6fb3f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:40.329859 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e114dc2ea54427285f383b99fe6fb3f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:40.330465 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e114dc2ea54427285f383b99fe6fb3f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:40.337386 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e114dc2ea54427285f383b99fe6fb3f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:40.469811 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e114dc2ea54427285f383b99fe6fb3f'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:40.488003 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1431/2845] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:02:39 2022] DELETE /identity/v3/users/7e114dc2ea54427285f383b99fe6fb3f => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:40.717384 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:40.728734 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217be8-b571-4e2f-b4f2-54a50c5277f0 service placement] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:40.788896 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:40.789798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:40.790075 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:40.790476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] PATH_INFO: `/v3/projects/38ede1d66d4549a7bf002db641fa11f2` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:40.791654 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ede1d66d4549a7bf002db641fa11f2)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:40.793583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:40.926990 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ede1d66d4549a7bf002db641fa11f2', 'request_id': 'req-58d139f3-756c-4fe1-8067-67b7a7db9f5a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:40.932364 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ede1d66d4549a7bf002db641fa11f2', 'request_id': 'req-58d139f3-756c-4fe1-8067-67b7a7db9f5a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:40.938956 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ede1d66d4549a7bf002db641fa11f2', 'request_id': 'req-58d139f3-756c-4fe1-8067-67b7a7db9f5a'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.058018 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] The token cache is being invalidate because project 38ede1d66d4549a7bf002db641fa11f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:02:41.058243 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.058837 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.060479 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1415/2846] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:02:40 2022] DELETE /identity/v3/projects/38ede1d66d4549a7bf002db641fa11f2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:41.064470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:41.065984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:41.079132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f92cf-56ba-4302-b344-acfd3801e06e admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:41.135455 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:41.136385 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:41.136699 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:41.136979 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] PATH_INFO: `/v3/users/39862ae401ac4891b524d6546dab4790` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:41.138178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39862ae401ac4891b524d6546dab4790)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:41.140094 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:41.161571 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:41.426586 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39862ae401ac4891b524d6546dab4790'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.427238 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39862ae401ac4891b524d6546dab4790'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.427832 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39862ae401ac4891b524d6546dab4790'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.434730 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39862ae401ac4891b524d6546dab4790'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.538722 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-187eb64e-9236-4f08-9bf7-18f1703bf2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39862ae401ac4891b524d6546dab4790'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:41.557343 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1432/2847] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:02:41 2022] DELETE /identity/v3/users/39862ae401ac4891b524d6546dab4790 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:41.832736 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:41.834285 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:41.847208 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d139f3-756c-4fe1-8067-67b7a7db9f5a admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:41.865209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:41.905027 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:41.905887 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:41.906169 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:41.906445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] PATH_INFO: `/v3/projects/7e23becce79241bd85f6b602a5e0b08f` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:41.907613 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e23becce79241bd85f6b602a5e0b08f)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:41.909646 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.047301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e23becce79241bd85f6b602a5e0b08f', 'request_id': 'req-e2d428eb-8f88-48d4-ad53-dae17f79131b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:42.053453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e23becce79241bd85f6b602a5e0b08f', 'request_id': 'req-e2d428eb-8f88-48d4-ad53-dae17f79131b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:42.059496 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e23becce79241bd85f6b602a5e0b08f', 'request_id': 'req-e2d428eb-8f88-48d4-ad53-dae17f79131b'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:42.160914 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] The token cache is being invalidate because project 7e23becce79241bd85f6b602a5e0b08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:02:42.161206 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:42.161845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-e2d428eb-8f88-48d4-ad53-dae17f79131b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:02:42.163372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1416/2848] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:02:41 2022] DELETE /identity/v3/projects/7e23becce79241bd85f6b602a5e0b08f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:42.168330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3542186-a909-4a41-af81-038dd9f9b894 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.170917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3542186-a909-4a41-af81-038dd9f9b894 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.170917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3542186-a909-4a41-af81-038dd9f9b894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.187222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c3542186-a909-4a41-af81-038dd9f9b894 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:42.188409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c3542186-a909-4a41-af81-038dd9f9b894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:42.246407 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1433/2849] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 18:02:42.254741 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62ca3e-ce89-484b-af73-186f0c8ea676 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.255033 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62ca3e-ce89-484b-af73-186f0c8ea676 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.255315 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62ca3e-ce89-484b-af73-186f0c8ea676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.273820 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-1f62ca3e-ce89-484b-af73-186f0c8ea676 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:42.275027 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1f62ca3e-ce89-484b-af73-186f0c8ea676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:42.282850 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1417/2850] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:42.287409 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3542186-a909-4a41-af81-038dd9f9b894 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:42.296839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3542186-a909-4a41-af81-038dd9f9b894 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:42.301453 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:42.302046 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.302330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.302607 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.303330 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80584) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 18:02:42.303635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:42.305326 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.317111 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1434/2851] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 18:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 18:02:42.321159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62ca3e-ce89-484b-af73-186f0c8ea676 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:42.330536 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62ca3e-ce89-484b-af73-186f0c8ea676 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:42.335634 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:42.336279 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.336722 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.337002 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.337693 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:42.339570 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.419980 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1418/2852] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:02:42 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:42.424141 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:42.434474 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9ba23-4b64-4484-8089-7ecdfd356ea7 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:42.489566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:42.490228 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.490516 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.490794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] PATH_INFO: `/v3/users` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.491488 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:42.493418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.629809 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1435/2853] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:02:42 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:02:42.633936 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:42.644677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9978bd-2568-45a2-90db-ac228c914ed0 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:42.649209 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:42.649862 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.650141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.650410 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.651127 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:42.652762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.663562 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1419/2854] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:42.667562 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:42.677892 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7853344-68ba-449a-bb7b-f444d19aaa39 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:42.682369 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93564f84-60b5-4b93-9754-bda9aeff621c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:42.683014 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93564f84-60b5-4b93-9754-bda9aeff621c admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.683297 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93564f84-60b5-4b93-9754-bda9aeff621c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.683569 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93564f84-60b5-4b93-9754-bda9aeff621c admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.684283 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93564f84-60b5-4b93-9754-bda9aeff621c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:42.685882 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93564f84-60b5-4b93-9754-bda9aeff621c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.696011 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1436/2855] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:02:42.700809 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:42.711154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1715-9702-45a7-bfc5-cacd88e04975 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:42.715843 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:42.716539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.716827 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.717102 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] PATH_INFO: `/v3/projects/8a09caa450f94f3ea5b9ebff8d206304/users/c54faad43103407ca99cedf7b8508144/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.731552 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a09caa450f94f3ea5b9ebff8d206304, user_id=c54faad43103407ca99cedf7b8508144, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:42.734269 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.807803 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1420/2856] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:02:42 2022] PUT /identity/v3/projects/8a09caa450f94f3ea5b9ebff8d206304/users/c54faad43103407ca99cedf7b8508144/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:02:42.812819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272034-f8a5-4919-9750-48e3b6774f0b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.813246 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272034-f8a5-4919-9750-48e3b6774f0b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.813552 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272034-f8a5-4919-9750-48e3b6774f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.820868 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:42.831779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-bc272034-f8a5-4919-9750-48e3b6774f0b None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:42.833020 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-bc272034-f8a5-4919-9750-48e3b6774f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:42.891838 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1437/2857] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:42.901257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:42.903035 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:42.953581 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:42.959539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22dc6e-f6ad-4d8c-a9c9-88bcc05db0d2 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:42.963167 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:42.963798 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:42.964104 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:42.968088 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:42.973824 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:42.975240 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:42.985919 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1421/2858] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:44.373758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db308d-a306-4475-a797-797aaf32c1da None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:44.373758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db308d-a306-4475-a797-797aaf32c1da None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:44.373758 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db308d-a306-4475-a797-797aaf32c1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:44.395220 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-73db308d-a306-4475-a797-797aaf32c1da None None] MFA Rules not processed for user `c54faad43103407ca99cedf7b8508144`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:44.396421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-73db308d-a306-4475-a797-797aaf32c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:44.462399 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1438/2859] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:44.473108 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:02:44.474523 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:44.530468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4aec-b938-4491-90b5-d7565a5ac200 service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:02:44.534751 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc474855-b354-4deb-87fe-5b0f59308a1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:02:44.535393 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc474855-b354-4deb-87fe-5b0f59308a1f service cinder] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:44.535677 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc474855-b354-4deb-87fe-5b0f59308a1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:44.535954 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc474855-b354-4deb-87fe-5b0f59308a1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:44.544372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc474855-b354-4deb-87fe-5b0f59308a1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:02:44.545963 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc474855-b354-4deb-87fe-5b0f59308a1f service cinder] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:02:44.554797 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1422/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:02:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:02:46.509226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383a4b0-6cf6-462b-9c57-8ebfa6a01715 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:02:46.509771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383a4b0-6cf6-462b-9c57-8ebfa6a01715 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:02:46.509771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383a4b0-6cf6-462b-9c57-8ebfa6a01715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:02:46.527043 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-3383a4b0-6cf6-462b-9c57-8ebfa6a01715 None None] MFA Rules not processed for user `c54faad43103407ca99cedf7b8508144`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:02:46.528241 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-3383a4b0-6cf6-462b-9c57-8ebfa6a01715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:02:46.535669 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1439/2861] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:02:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:02:54.550882 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=31 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:02:56.521959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=60 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:08.313478 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:08.313972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:08.314084 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:08.331661 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] MFA Rules not processed for user `c54faad43103407ca99cedf7b8508144`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:08.332901 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:08.339702 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1423/2862] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:03:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:18.327043 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=3 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:31.794877 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:31.795445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:31.795445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:31.854955 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:31.856170 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:31.924575 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1440/2863] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Jul 20 18:03:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:32.200519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:32.202214 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:32.302567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f8dd3-ccfd-440c-9c82-c6d8fe55ac5a None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:32.308139 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1d966-0c0b-4636-82f6-257652133bd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:32.308811 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d966-0c0b-4636-82f6-257652133bd8 service swift] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:32.309094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d966-0c0b-4636-82f6-257652133bd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:32.309372 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d966-0c0b-4636-82f6-257652133bd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:32.318689 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1d966-0c0b-4636-82f6-257652133bd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:32.320604 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1d966-0c0b-4636-82f6-257652133bd8 service swift] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:32.330917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1424/2864] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 18:03:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 18:03:40.742411 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:40.743932 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:40.830299 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86324541-3f59-4511-a8f8-5afb4946797b None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:40.836525 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:40.846445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:40.846445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:40.846445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:40.846445 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:40.852916 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:40.854685 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:40.867012 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1441/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:03:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:03:42.326553 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=41 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:50.166686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9cee8-dea0-4130-b973-bf2c6ddd2a2e None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:50.167257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9cee8-dea0-4130-b973-bf2c6ddd2a2e None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:50.167257 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9cee8-dea0-4130-b973-bf2c6ddd2a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:50.185444 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-68f9cee8-dea0-4130-b973-bf2c6ddd2a2e None None] MFA Rules not processed for user `e3d36cdf82db411e9de1d2f091c16e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:50.186617 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-68f9cee8-dea0-4130-b973-bf2c6ddd2a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:50.193917 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1425/2866] 10.222.0.42 () {58 vars in 967 bytes} [Wed Jul 20 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:50.207024 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:50.219055 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a87a87-c5e7-46e6-beb2-2950c516c59d service placement] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:50.226588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2335e75507f44afb8d77f9b70a668a94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:50.227446 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:50.227949 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:50.228470 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:50.239930 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:50.242021 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:50.252621 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1442/2867] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Jul 20 18:03:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 18:03:54.643233 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9cee8-dea0-4130-b973-bf2c6ddd2a2e None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:54.661131 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9cee8-dea0-4130-b973-bf2c6ddd2a2e None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:54.666413 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:54.667303 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:54.667584 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:54.667859 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] PATH_INFO: `/v3/users/c54faad43103407ca99cedf7b8508144` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:54.669116 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c54faad43103407ca99cedf7b8508144)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:54.671087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:54.907521 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c54faad43103407ca99cedf7b8508144'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:54.920758 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c54faad43103407ca99cedf7b8508144'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:54.921499 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c54faad43103407ca99cedf7b8508144'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:54.922106 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c54faad43103407ca99cedf7b8508144'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:54.929067 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-c3e07379-bdce-4fb7-9ff3-319ab9739f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c54faad43103407ca99cedf7b8508144'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:55.127816 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1426/2868] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:03:54 2022] DELETE /identity/v3/users/c54faad43103407ca99cedf7b8508144 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:03:55.568780 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:55.581147 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbad0d-561f-41bb-a87d-c5c8f28303f5 service swift] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:55.622674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=100 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:55.641751 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:55.642678 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:55.642965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:55.643336 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] PATH_INFO: `/v3/projects/8a09caa450f94f3ea5b9ebff8d206304` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:55.644958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a09caa450f94f3ea5b9ebff8d206304)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:55.646984 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:55.865835 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a09caa450f94f3ea5b9ebff8d206304', 'request_id': 'req-94098f97-7b64-4a45-a114-53c380dda745'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:55.872306 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a09caa450f94f3ea5b9ebff8d206304', 'request_id': 'req-94098f97-7b64-4a45-a114-53c380dda745'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:56.075462 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a09caa450f94f3ea5b9ebff8d206304', 'request_id': 'req-94098f97-7b64-4a45-a114-53c380dda745'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:56.088958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] The token cache is being invalidate because project 8a09caa450f94f3ea5b9ebff8d206304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:03:56.089400 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:56.092037 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-94098f97-7b64-4a45-a114-53c380dda745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:56.092252 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1443/2869] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:03:55 2022] DELETE /identity/v3/projects/8a09caa450f94f3ea5b9ebff8d206304 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:03:56.097414 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912aba-c992-4221-8bd5-e6951959ef4c None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.098141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912aba-c992-4221-8bd5-e6951959ef4c None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.098141 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912aba-c992-4221-8bd5-e6951959ef4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.117204 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-b1912aba-c992-4221-8bd5-e6951959ef4c None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:56.118453 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b1912aba-c992-4221-8bd5-e6951959ef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:56.181648 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1427/2870] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:56.190740 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe7ac5-59bd-4449-b73c-ca838df6ebd3 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.191030 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe7ac5-59bd-4449-b73c-ca838df6ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.191309 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe7ac5-59bd-4449-b73c-ca838df6ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.210382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-debe7ac5-59bd-4449-b73c-ca838df6ebd3 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:56.211628 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-debe7ac5-59bd-4449-b73c-ca838df6ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:56.219853 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1444/2871] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:56.224228 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912aba-c992-4221-8bd5-e6951959ef4c None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:56.234154 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912aba-c992-4221-8bd5-e6951959ef4c None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:56.239426 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:56.240093 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.240379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.240686 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.241483 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 18:03:56.241795 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:56.243441 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:56.256422 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1428/2872] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 18:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 18:03:56.267356 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe7ac5-59bd-4449-b73c-ca838df6ebd3 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:56.279115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe7ac5-59bd-4449-b73c-ca838df6ebd3 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:56.284265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:56.284959 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.285288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.285766 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.286595 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:56.288590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:56.431610 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1445/2873] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:03:56 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:03:56.436051 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:56.446996 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40b9fe-bfd7-427f-81b3-9d88d4b99656 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:56.503078 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:56.512328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:56.513112 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.513368 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.513644 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.514375 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:56.516328 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:56.661120 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1429/2874] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:03:56 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:03:56.665913 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:56.677774 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e9d0-2ee7-435f-b473-0c8e4cbeb870 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:56.686421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:56.686421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.686421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.686421 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.686937 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:56.687861 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:56.702483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1446/2875] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 18:03:56.706576 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:56.718087 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb75f22-d59d-441d-a165-23bd92a77b18 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:56.723991 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f26eef-d80e-4eb4-8d92-aa0b854f3137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:56.725070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f26eef-d80e-4eb4-8d92-aa0b854f3137 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.725070 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f26eef-d80e-4eb4-8d92-aa0b854f3137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.725369 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f26eef-d80e-4eb4-8d92-aa0b854f3137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.726110 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f26eef-d80e-4eb4-8d92-aa0b854f3137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:56.727771 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f26eef-d80e-4eb4-8d92-aa0b854f3137 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:56.739891 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1430/2876] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:03:56.744483 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:56.755115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfd903-9b78-42f9-9635-09c38bbfc6de admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:56.759788 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:56.760746 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.761061 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.761308 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] PATH_INFO: `/v3/projects/dd606c9cbd954c46b89e70a15cf763ba/users/be3f65c74f85407fa253a3033eb0babd/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.779643 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd606c9cbd954c46b89e70a15cf763ba, user_id=be3f65c74f85407fa253a3033eb0babd, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:56.782428 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:56.913206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1447/2877] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:03:56 2022] PUT /identity/v3/projects/dd606c9cbd954c46b89e70a15cf763ba/users/be3f65c74f85407fa253a3033eb0babd/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:03:56.923601 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1090e-2a3f-4165-97cf-ec9d86c53908 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:56.924001 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1090e-2a3f-4165-97cf-ec9d86c53908 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:56.924353 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1090e-2a3f-4165-97cf-ec9d86c53908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:56.953846 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-bef1090e-2a3f-4165-97cf-ec9d86c53908 None None] MFA Rules not processed for user `be3f65c74f85407fa253a3033eb0babd`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:56.955714 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-bef1090e-2a3f-4165-97cf-ec9d86c53908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:57.044505 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1431/2878] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:03:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:57.054047 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:57.058613 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:57.142629 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72751b-6411-4102-bd70-dd9962bd5a4d admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:57.147284 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f21e7-fb64-47e6-9fb7-9bc518617d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:57.148226 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f21e7-fb64-47e6-9fb7-9bc518617d7d service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:57.150819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f21e7-fb64-47e6-9fb7-9bc518617d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:57.150819 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f21e7-fb64-47e6-9fb7-9bc518617d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:57.160771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f21e7-fb64-47e6-9fb7-9bc518617d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:57.162684 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f21e7-fb64-47e6-9fb7-9bc518617d7d service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:57.185794 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1448/2879] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:03:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:03:57.201068 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1090e-2a3f-4165-97cf-ec9d86c53908 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:57.276583 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1090e-2a3f-4165-97cf-ec9d86c53908 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:57.289748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:57.289748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:57.289748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:57.289748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] PATH_INFO: `/v3/users/be3f65c74f85407fa253a3033eb0babd` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:57.293874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3f65c74f85407fa253a3033eb0babd)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:57.293874 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:57.608789 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3f65c74f85407fa253a3033eb0babd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:57.630504 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:57.632378 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3f65c74f85407fa253a3033eb0babd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:57.633330 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3f65c74f85407fa253a3033eb0babd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:57.634048 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3f65c74f85407fa253a3033eb0babd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:57.643239 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3f65c74f85407fa253a3033eb0babd'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:57.864293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1432/2880] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:03:57 2022] DELETE /identity/v3/users/be3f65c74f85407fa253a3033eb0babd => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:03:57.866353 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:57.866840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:57.866939 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:57.886712 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:57.887294 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:57.891590 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:57.958840 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1449/2881] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:57.971038 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:57.973300 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:58.046629 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5377ced-6081-4c4d-9087-b4e5d05318c4 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:58.050807 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e785097-5cad-4db4-8e05-6e6911620913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:58.051379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785097-5cad-4db4-8e05-6e6911620913 service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:58.051738 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785097-5cad-4db4-8e05-6e6911620913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:58.052095 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785097-5cad-4db4-8e05-6e6911620913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:58.062897 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e785097-5cad-4db4-8e05-6e6911620913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:58.064899 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e785097-5cad-4db4-8e05-6e6911620913 service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:58.085015 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1433/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:03:58.734957 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:58.738395 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:58.753625 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5bcd2-9f4c-42c8-ab55-6e1d57d938d0 None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:58.759089 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:58.760177 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:58.760508 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:58.760791 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] PATH_INFO: `/v3/projects/dd606c9cbd954c46b89e70a15cf763ba` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:58.762242 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd606c9cbd954c46b89e70a15cf763ba)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:58.764363 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:58.926286 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd606c9cbd954c46b89e70a15cf763ba', 'request_id': 'req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:58.929545 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd606c9cbd954c46b89e70a15cf763ba', 'request_id': 'req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:59.052686 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd606c9cbd954c46b89e70a15cf763ba', 'request_id': 'req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:59.067062 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] The token cache is being invalidate because project dd606c9cbd954c46b89e70a15cf763ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80584) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:03:59.067350 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:59.067999 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-6693d06e-bfbd-4bed-aa71-0ea6f40ab3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:03:59.069600 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1450/2883] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:03:58 2022] DELETE /identity/v3/projects/dd606c9cbd954c46b89e70a15cf763ba => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:03:59.074858 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a3c4-b3d8-4f8c-8b21-a8c4937dcfd1 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.075344 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a3c4-b3d8-4f8c-8b21-a8c4937dcfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.075456 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a3c4-b3d8-4f8c-8b21-a8c4937dcfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.093134 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-e730a3c4-b3d8-4f8c-8b21-a8c4937dcfd1 None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:59.094366 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e730a3c4-b3d8-4f8c-8b21-a8c4937dcfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:59.163762 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1434/2884] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:59.171849 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801868e-4bc0-4394-b02b-808647b28d1f None None] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.172145 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801868e-4bc0-4394-b02b-808647b28d1f None None] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.172450 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801868e-4bc0-4394-b02b-808647b28d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.192288 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.auth.core [None req-4801868e-4bc0-4394-b02b-808647b28d1f None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80584) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:03:59.193635 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-4801868e-4bc0-4394-b02b-808647b28d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:03:59.202854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1451/2885] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:03:59.207933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730a3c4-b3d8-4f8c-8b21-a8c4937dcfd1 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:59.218166 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730a3c4-b3d8-4f8c-8b21-a8c4937dcfd1 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:59.223527 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:59.224187 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.224501 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.224782 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] PATH_INFO: `/v3/domains` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.225559 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80583) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 18:03:59.225828 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:59.227435 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:59.239361 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1435/2886] 10.222.0.42 () {58 vars in 1152 bytes} [Wed Jul 20 18:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 18:03:59.243960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801868e-4bc0-4394-b02b-808647b28d1f None None] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:59.253609 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801868e-4bc0-4394-b02b-808647b28d1f None None] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:59.262376 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:59.263085 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] REQUEST_METHOD: `POST` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.263370 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.263646 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.264346 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:59.266771 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:59.410328 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1452/2887] 10.222.0.42 () {60 vars in 1152 bytes} [Wed Jul 20 18:03:59 2022] POST /identity/v3/projects => generated 391 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 18:03:59.414307 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:59.424692 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc606031-c498-4e85-8451-342f57b44642 admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:59.486301 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:59.486951 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.487238 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.487514 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.488210 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:59.490320 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:59.653730 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:03:59.660039 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1436/2888] 10.222.0.42 () {60 vars in 1143 bytes} [Wed Jul 20 18:03:59 2022] POST /identity/v3/users => generated 296 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 18:03:59.666821 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:59.675459 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f128bf-0fb2-4374-bd3d-0188785b3977 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:59.680278 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:59.681018 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.681373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.681654 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.682435 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:59.684091 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:59.695566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1453/2889] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:03:59.700197 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:59.710476 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cb3ce-d40e-464f-95b7-bcea5aa38ebd admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:59.715449 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc1c4-3948-4ffc-8aee-c7c3330b5efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:59.726495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc1c4-3948-4ffc-8aee-c7c3330b5efc admin admin] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.726495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc1c4-3948-4ffc-8aee-c7c3330b5efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.726495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc1c4-3948-4ffc-8aee-c7c3330b5efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.726495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cc1c4-3948-4ffc-8aee-c7c3330b5efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:59.726495 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cc1c4-3948-4ffc-8aee-c7c3330b5efc admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:03:59.733819 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1437/2890] 10.222.0.42 () {58 vars in 1121 bytes} [Wed Jul 20 18:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 18:03:59.738657 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:03:59.748884 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122b146-aa3f-4148-9b37-7b35eb08f3ac admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:03:59.753397 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:03:59.754277 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:03:59.754570 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:03:59.754845 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] PATH_INFO: `/v3/projects/b38d189c2a834c04a02b4f8c645e6217/users/06806e13594448c48285cb2f1dda1d87/roles/295c91f6f2554a0ea6f954d898f9725c` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:03:59.771676 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38d189c2a834c04a02b4f8c645e6217, user_id=06806e13594448c48285cb2f1dda1d87, role_id=295c91f6f2554a0ea6f954d898f9725c)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:03:59.774467 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:00.001650 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1454/2891] 10.222.0.42 () {60 vars in 1482 bytes} [Wed Jul 20 18:03:59 2022] PUT /identity/v3/projects/b38d189c2a834c04a02b4f8c645e6217/users/06806e13594448c48285cb2f1dda1d87/roles/295c91f6f2554a0ea6f954d898f9725c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:04:00.013557 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:00.013847 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:00.014202 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:00.044064 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] MFA Rules not processed for user `06806e13594448c48285cb2f1dda1d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:04:00.045354 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:00.123021 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1438/2892] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:04:00.134674 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:00.136351 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:00.169567 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:04:00.198638 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2b4e-9941-411c-8c04-853238606b84 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:00.202487 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9706ef808a224c83aca165c595b596ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:00.203135 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:00.203419 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:00.203697 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:00.213564 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:00.215183 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:00.225202 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1455/2893] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 18:04:08.404993 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:08.406567 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:08.467710 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f30117-14a6-40c1-bcd7-cdaf8c7389c0 None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:08.471825 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:08.472489 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] REQUEST_METHOD: `GET` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:08.472775 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:08.473046 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:08.474863 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:08.537985 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:08.539539 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:08.549791 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1439/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:04:10.949282 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=13 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:04:16.515960 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:16.518310 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:16.537398 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015b273-ae1b-43bc-b366-f062d558f624 service cinder] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:16.544790 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:16.545748 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:16.545958 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:16.546231 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] PATH_INFO: `/v3/users/9c95e9df33ad4f819f26fb8b695df7e9` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:16.547452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c95e9df33ad4f819f26fb8b695df7e9)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:16.550787 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:16.840169 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c95e9df33ad4f819f26fb8b695df7e9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:16.840965 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c95e9df33ad4f819f26fb8b695df7e9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:16.841664 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c95e9df33ad4f819f26fb8b695df7e9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:16.850184 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c95e9df33ad4f819f26fb8b695df7e9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:16.955101 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c95e9df33ad4f819f26fb8b695df7e9'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:16.978216 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1456/2895] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:04:16 2022] DELETE /identity/v3/users/9c95e9df33ad4f819f26fb8b695df7e9 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:04:17.330331 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:17.343642 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86664ff-1218-4c76-9fa1-662ac1cb279f service neutron] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:17.390519 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=100 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:04:17.437845 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:17.439403 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:17.440071 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:17.440513 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] PATH_INFO: `/v3/projects/be1213650f38494aa2aeb1aadc7ccbc7` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:17.442211 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1213650f38494aa2aeb1aadc7ccbc7)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:17.444190 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:17.612528 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1213650f38494aa2aeb1aadc7ccbc7', 'request_id': 'req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:17.618171 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1213650f38494aa2aeb1aadc7ccbc7', 'request_id': 'req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:17.631254 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1213650f38494aa2aeb1aadc7ccbc7', 'request_id': 'req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:17.772445 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] The token cache is being invalidate because project be1213650f38494aa2aeb1aadc7ccbc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:04:17.772912 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:17.773754 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:17.775972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1440/2896] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:04:17 2022] DELETE /identity/v3/projects/be1213650f38494aa2aeb1aadc7ccbc7 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:04:17.780217 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:17.781727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:17.796636 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6562e33-3fc4-4662-9d52-48a7cbc96e34 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:17.881588 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:17.882561 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:17.883060 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:17.883439 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] PATH_INFO: `/v3/users/ef540a5f6a9b4a4980f69d0553e5b058` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:17.885182 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef540a5f6a9b4a4980f69d0553e5b058)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:17.888382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:18.262452 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef540a5f6a9b4a4980f69d0553e5b058'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:18.263222 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef540a5f6a9b4a4980f69d0553e5b058'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:18.264132 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef540a5f6a9b4a4980f69d0553e5b058'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:18.272382 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef540a5f6a9b4a4980f69d0553e5b058'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:18.433865 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef540a5f6a9b4a4980f69d0553e5b058'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:18.443303 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:04:18.454115 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1457/2897] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:04:17 2022] DELETE /identity/v3/users/ef540a5f6a9b4a4980f69d0553e5b058 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:04:18.787118 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:18.789079 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:18.805895 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340c800-7fd3-4e25-ba4d-a4e6ae226deb admin admin] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:18.882972 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:18.883871 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:18.884159 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:18.884454 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] PATH_INFO: `/v3/projects/b4291ba018c443d392571201d810b551` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:18.885764 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4291ba018c443d392571201d810b551)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:18.887804 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:19.430034 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4291ba018c443d392571201d810b551', 'request_id': 'req-45e4834e-d9fa-455c-a841-bd7d9f5544f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:19.435580 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4291ba018c443d392571201d810b551', 'request_id': 'req-45e4834e-d9fa-455c-a841-bd7d9f5544f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:19.437468 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:04:19.442748 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4291ba018c443d392571201d810b551', 'request_id': 'req-45e4834e-d9fa-455c-a841-bd7d9f5544f3'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:19.686589 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] The token cache is being invalidate because project b4291ba018c443d392571201d810b551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:04:19.686933 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:19.687544 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-45e4834e-d9fa-455c-a841-bd7d9f5544f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:19.689703 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1441/2898] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:04:18 2022] DELETE /identity/v3/projects/b4291ba018c443d392571201d810b551 => generated 0 bytes in 903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:04:22.919319 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:22.920799 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:22.934179 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c417-4c85-474a-b5f6-5e4836fae70c admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:22.991265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:22.992133 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:22.992418 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:22.992727 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] PATH_INFO: `/v3/users/06806e13594448c48285cb2f1dda1d87` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:22.993917 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06806e13594448c48285cb2f1dda1d87)` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:22.995839 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:23.218173 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06806e13594448c48285cb2f1dda1d87'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:23.219378 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06806e13594448c48285cb2f1dda1d87'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:23.220854 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06806e13594448c48285cb2f1dda1d87'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:23.228677 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06806e13594448c48285cb2f1dda1d87'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:23.343721 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.notifications [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06806e13594448c48285cb2f1dda1d87'} {{(pid=80584) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:23.363373 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1458/2899] 10.222.0.42 () {58 vars in 1223 bytes} [Wed Jul 20 18:04:22 2022] DELETE /identity/v3/users/06806e13594448c48285cb2f1dda1d87 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:04:23.368094 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] REQUEST_METHOD: `POST` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:23.368675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:23.368675 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:23.387474 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.auth.core [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] MFA Rules not processed for user `85af2924b4aa4547ad22ecda72bbb229`. Rule list: `[]` (Enabled: `True`). {{(pid=80583) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 18:04:23.388962 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:23.451293 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1442/2900] 10.222.0.42 () {58 vars in 957 bytes} [Wed Jul 20 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 18:04:23.462519 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:23.464524 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:23.533507 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bf2d-6e81-467a-bc62-a59afc579f81 admin admin] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:23.538834 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1d18a38f45e4501914e22689f145fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:23.539537 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:23.539820 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:23.540096 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:23.553123 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:23.554566 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:23.565437 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1459/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 18:04:24.032631 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] Authenticating user token {{(pid=80583) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:04:24.038582 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.fernet_utils [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80583) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:04:24.050538 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdef91-2247-4598-94f7-082ca55332ad None None] Validating token access rules against request {{(pid=80583) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:04:24.055333 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85af2924b4aa4547ad22ecda72bbb229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ddf29534d9849ed9221c0b30b65b542', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80583) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:04:24.056200 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:04:24.056511 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:04:24.056790 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] PATH_INFO: `/v3/projects/b38d189c2a834c04a02b4f8c645e6217` {{(pid=80583) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:04:24.058161 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38d189c2a834c04a02b4f8c645e6217)` {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:04:24.060423 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] RBAC: Authorization granted {{(pid=80583) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:04:24.166706 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38d189c2a834c04a02b4f8c645e6217', 'request_id': 'req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:24.172379 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38d189c2a834c04a02b4f8c645e6217', 'request_id': 'req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:24.178932 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38d189c2a834c04a02b4f8c645e6217', 'request_id': 'req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0'} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:24.263510 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] The token cache is being invalidate because project b38d189c2a834c04a02b4f8c645e6217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80583) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 18:04:24.263786 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:24.264452 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG keystone.notifications [None req-9bc9b2a6-8f6a-49c1-ac48-298a79f077c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80583) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 18:04:24.266024 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: [pid: 80583|app: 0|req: 1443/2902] 10.222.0.42 () {58 vars in 1232 bytes} [Wed Jul 20 18:04:24 2022] DELETE /identity/v3/projects/b38d189c2a834c04a02b4f8c645e6217 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 18:04:33.561068 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=82,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:04:34.259875 dv-cinder-849846-3-iscsi devstack@keystone.service[80583]: DEBUG dbcounter [-] [80583] Writing DB stats keystone:SELECT=57,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80583) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 18:08:42.675779 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] Authenticating user token {{(pid=80584) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 18:08:42.677258 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:08:42.815010 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691082e-7134-41ad-804a-17daee5c4eb8 service neutron] Validating token access rules against request {{(pid=80584) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 18:08:42.884206 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b4ca7-f2d8-4145-8018-c73fff7f090f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4871b13f87f1498ba33f8ecd2a6c0e39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4360ee3455064eea9e0b5197debd9a8f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80584) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 18:08:42.884887 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b4ca7-f2d8-4145-8018-c73fff7f090f service placement] REQUEST_METHOD: `GET` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 18:08:42.885178 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b4ca7-f2d8-4145-8018-c73fff7f090f service placement] SCRIPT_NAME: `/identity` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 18:08:42.885458 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b4ca7-f2d8-4145-8018-c73fff7f090f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80584) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 18:08:42.887265 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.fernet_utils [None req-295b4ca7-f2d8-4145-8018-c73fff7f090f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80584) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 18:08:42.899878 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b4ca7-f2d8-4145-8018-c73fff7f090f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 18:08:42.901503 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b4ca7-f2d8-4145-8018-c73fff7f090f service placement] RBAC: Authorization granted {{(pid=80584) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 18:08:42.911681 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: [pid: 80584|app: 0|req: 1460/2903] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Jul 20 18:08:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 18:08:52.907243 dv-cinder-849846-3-iscsi devstack@keystone.service[80584]: DEBUG dbcounter [-] [80584] Writing DB stats keystone:SELECT=89 {{(pid=80584) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}